Reframing Atlas
A curated, citable dataset and dashboard of multi-agent attack scenarios and empirical compliance results across model families, built for safety researchers and policy advisors writing AI governance briefs.
AI safety researchers and policy advisors writing governance briefs
- Browseable scenarios tagged by reframing, delegation framing, and planner refusal
- Side-by-side compliance results across GPT, Gemini, Claude, DeepSeek
- Exportable charts and CSVs ready for policy reports and academic citations
- Quarterly refresh as new model versions ship
The paper explicitly argues multi-agent evaluations must report the four mechanisms separately before attributing failures to architecture — a tooling gap governance work is begging to be filled.
Multiple active 2024-2025 benchmarks (TAMAS with 300 adversarial instances across 10 LLMs, HAJailBench with 11,100 labeled interactions, MultiAgentBench) confirm real research interest, but the narrow 'policy advisor writing governance briefs' segment is unproven as paying users.TAMAS: Benchmarking Adversarial Risks in Multi-Agent LLM Systems ↗Efficient LLM Safety Evaluation through Multi-Agent Debate (HAJailBench) ↗
Direct competitors exist: TAMAS (multi-agent adversarial benchmark, Nov 2025), AILuminate (MLCommons, 24k prompts, 12 hazard categories), phoenix-assistant/ai-governance-tracker ('Bloomberg Terminal for AI safety'), and Microsoft Security Dashboard for AI — the 'citable curated dataset' angle for multi-agent attacks is being actively filled.TAMAS: Benchmarking Adversarial Risks in Multi-Agent LLM Systems ↗AI Governance Tracker (Bloomberg Terminal for AI safety) ↗AILuminate - MLCommons ↗
Academic safety datasets (TAMAS, AILuminate, SafetyBench, HAJailBench) are released free on arXiv/GitHub; the phoenix-assistant governance tracker is open-source; policy advisors in think tanks/NGOs/governments have small budgets and rely on free public sources — no evidence of willingness to pay for research datasets.Awesome Agent Benchmarks (GitHub) ↗MLCommons AILuminate v1.1 benchmark suite ↗
Multi-agent safety is a rapidly expanding field with EU AI Act, NIST, and emerging regulation creating durable demand for governance evidence; the four-mechanism separation debate is an open methodological question that will take years to settle.Red-Teaming LLM Multi-Agent Systems via Communication Attacks ↗Efficient LLM Safety Evaluation through Multi-Agent Debate ↗
Curation from existing papers is straightforward, but a live dashboard with fresh empirical compliance runs across model families requires substantial compute, API spend, and ongoing maintenance; the 'citable' bar implies peer-review-grade methodology that a small team would struggle to sustain.TAMAS: Benchmarking Adversarial Risks in Multi-Agent LLM Systems ↗