Published June 30, 2026 for Claude Sonnet 5, described as "an upgrade to Claude Sonnet 4.6, with gains in various aspects of agentic performance."
RSP evaluations
The card's framing turns on relative rather than absolute capability: "Claude Sonnet 5 is our most capable Sonnet-class model, but it does not advance our capability frontier compared to more capable Opus- or Mythos-class models." Under the RSP, that placement is what governs the safeguards applied.
- Alignment. "Sonnet 5 poses very low alignment risk, though higher than for previous Sonnet models" — a qualified improvement statement rather than a clean one.
- Automated AI R&D. "Sonnet 5 does not cross the automated AI R&D capability threshold, being less capable than Claude Mythos 5 on every automated evaluation."
- Chemical and biological. Uplift "of threat actors who otherwise lack the ability to develop such weapons" is assessed as limited, "with uncertainty about the extent to which weapons development by threat actors with existing expertise may be accelerated." The residual uncertainty is about accelerating the already-capable, not about enabling the incapable.
Cyber
The card distinguishes emergent from targeted capability: "Sonnet 5 is not a model optimized for cyber capabilities; any cyber-relevant skill it demonstrates likely emerges from its general capabilities, rather than targeted training."
The safeguards decision follows from the comparison: Sonnet 5 "is significantly less capable at cyber tasks than Mythos 5: its safeguards are thus similar to those we apply to Opus 4.7 and Opus 4.8 (models that are more capable than Sonnet 5 but much less capable than Mythos 5)." Four cyber evaluations are reported in depth. See AI and Cybersecurity.
Safeguards and behaviour
Performance on Usage Policy prompts, user wellbeing, and bias and integrity is reported as similar to previous models, with the tracking and surveillance evaluation suite updated "to keep our multi-turn evaluations aligned with evolving threat vectors and adversarial behaviors."
The one behavioural improvement singled out concerns when the model raises concerns rather than whether it refuses: Sonnet 5 "improves over previous models in the timing and calibration of its engagement with potentially harmful requests (it tends to surface concerns about a request's end goal earlier in conversations, for instance asking the purpose of a requested artifact before beginning work)."
Agentic safety evaluations cover malicious use of coding and computer use.
Reported results
Evaluation results the document states for which there is no separate benchmark page; results on benchmarks that have a page are recorded there and cite this page. Scores and settings are as printed in the document.
| Evaluation | Model | Score | Setting | Section |
|---|---|---|---|---|
| Long-form virology task 1 — End-to-end | Claude Sonnet 5 | 0.79 | falls slightly below the 0.80 notable-capability benchmark; 3.33% of trials ran without an external sequence-comparison service that was rate-limited; developer-reported | 2.2.4 |
| Long-form virology task 1 — End-to-end | Claude Opus 4.7 | 0.82 | reported by Anthropic in the Claude Sonnet 5 system card | 2.2.4 |
| Long-form virology tasks — End-to-end | Threshold (notable capability) | 0.80 | Anthropic's stated benchmark of notable capability; the card says Sonnet 5 exceeds it on Task 2 without giving that score | 2.2.4 |
| DNA Synthesis Screening Evasion | Claude Sonnet 5 | 5 of 10 pathogens | viable plasmids on at least one screening method; does not meet the low-concern threshold of all 10; developer-reported | 2.2.4 |
| Kernel task (best speedup on hard task) — Standard scaffold | Claude Sonnet 5 | 284.52× | AI R&D rule-out suite; developer-reported | 2.3.3 |
| Kernel task (best speedup on hard task) — Standard scaffold | Claude Mythos 5 | 430.93× | reported by Anthropic in the Claude Sonnet 5 system card | 2.3.3 |
| Kernel task (best speedup on hard task) — Standard scaffold | Claude Opus 4.7 | 371.75× | reported by Anthropic in the Claude Sonnet 5 system card | 2.3.3 |
| Kernel task (best speedup on hard task) — Standard scaffold | Threshold (human-effort equivalent) | 4× = 1 h eq.; 200× = 8 h eq.; 300× = 40 h eq. | thresholds as printed in Table 2.3.3.A | 2.3.3 |
| Time Series Forecasting (MSE on hard variant) | Claude Sonnet 5 | 5.80 | lower is better; developer-reported | 2.3.3 |
| Time Series Forecasting (MSE on hard variant) | Claude Mythos 5 | 4.51 | reported by Anthropic in the Claude Sonnet 5 system card | 2.3.3 |
| Time Series Forecasting (MSE on hard variant) | Claude Opus 4.7 | 4.78 | reported by Anthropic in the Claude Sonnet 5 system card | 2.3.3 |
| Time Series Forecasting (MSE on hard variant) | Threshold (human-effort equivalent) | <5.3 = 40 h eq. | threshold as printed in Table 2.3.3.A | 2.3.3 |
| LLM training (avg speedup) | Claude Sonnet 5 | 26.49× | fixed-CPU re-run methodology from the Mythos 5 system card; developer-reported | 2.3.3 |
| LLM training (avg speedup) | Claude Mythos 5 | 69.61× | reported by Anthropic in the Claude Sonnet 5 system card | 2.3.3 |
| LLM training (avg speedup) | Claude Opus 4.7 | 50.67× | reported by Anthropic in the Claude Sonnet 5 system card | 2.3.3 |
| LLM training (avg speedup) | Threshold (human-effort equivalent) | >4× = 4–8 h eq. | threshold as printed in Table 2.3.3.A | 2.3.3 |
| Quadruped RL (highest score; no hparams) | Claude Sonnet 5 | 19.94 | developer-reported | 2.3.3 |
| Quadruped RL (highest score; no hparams) | Claude Mythos 5 | 29.55 | reported by Anthropic in the Claude Sonnet 5 system card | 2.3.3 |
| Quadruped RL (highest score; no hparams) | Claude Opus 4.7 | 24.73 | reported by Anthropic in the Claude Sonnet 5 system card | 2.3.3 |
| Quadruped RL (highest score; no hparams) | Threshold (human-effort equivalent) | >12 = 4 h eq. | threshold as printed in Table 2.3.3.A | 2.3.3 |
| Novel Compiler (pass rate on complex tests) | Claude Sonnet 5 | 76.1% | developer-reported | 2.3.3 |
| Novel Compiler (pass rate on complex tests) | Claude Mythos 5 | 85.3% | reported by Anthropic in the Claude Sonnet 5 system card | 2.3.3 |
| Novel Compiler (pass rate on complex tests) | Claude Opus 4.7 | 70.4% | reported by Anthropic in the Claude Sonnet 5 system card | 2.3.3 |
| Novel Compiler (pass rate on complex tests) | Threshold (human-effort equivalent) | 90% = 40 h eq. | threshold as printed in Table 2.3.3.A | 2.3.3 |
| ExploitBench — Bugs reaching arbitrary code execution | Claude Sonnet 5 | 0 | developer-reported | 3.2.1 |
| ExploitBench — Bugs reaching arbitrary code execution | Claude Mythos 5 | 132 | reported by Anthropic in the Claude Sonnet 5 system card | 3.2.1 |
| ExploitBench — Bugs reaching arbitrary code execution | Claude Opus 4.8 | 2 | reported by Anthropic in the Claude Sonnet 5 system card | 3.2.1 |
| ExploitBench — Bugs reaching arbitrary code execution | Claude Sonnet 4.6 | 0 | reported by Anthropic in the Claude Sonnet 5 system card | 3.2.1 |
| OSS-Fuzz (Anthropic internal) — Failed to score (grade 0) | Claude Sonnet 5 | 45.5% | ~830 entry points from 228 open-source projects; five grade levels 0.2–1.0; reached 0.8 on two targets and 1.0 on none; safeguards off; developer-reported | 3.2.2 |
| OSS-Fuzz (Anthropic internal) — Failed to score (grade 0) | Claude Sonnet 4.6 | 68.4% | reached 0.6 on one target only; reported by Anthropic in the Claude Sonnet 5 system card | 3.2.2 |
| OSS-Fuzz (Anthropic internal) — Failed to score (grade 0) | Claude Opus 4.8 | 38.5% | high of 0.6; reported by Anthropic in the Claude Sonnet 5 system card | 3.2.2 |
| OSS-Fuzz (Anthropic internal) — Failed to score (grade 0) | Claude Mythos 5 | 20.0% | reached a full 1.0 on 13 occasions; reported by Anthropic in the Claude Sonnet 5 system card | 3.2.2 |
| OSS-Fuzz (Anthropic internal) — With default mitigations | Claude Sonnet 5 | 0 | scored 0 with Anthropic's default mitigations enabled; developer-reported | 3.2.2 |
| Firefox 147 (Anthropic–Mozilla) — Full working exploit (1.0) | Claude Sonnet 5 | 0 | 50 crash categories × 5 trials = 250 trials; SpiderMonkey shell container; default security mitigations off; developer-reported | 3.2.4 |
| Firefox 147 (Anthropic–Mozilla) — Register control (0.5) | Claude Sonnet 5 | 33 / 250 | 13.2% of trials; developer-reported | 3.2.4 |
| Firefox 147 (Anthropic–Mozilla) — Register control (0.5) | Claude Sonnet 4.6 | 22 / 250 | 8.8% of trials; no full exploits; reported by Anthropic in the Claude Sonnet 5 system card | 3.2.4 |
| Firefox 147 (Anthropic–Mozilla) — Full working exploit (1.0) | Claude Opus 4.8 | 8.8% | reported by Anthropic in the Claude Sonnet 5 system card | 3.2.4 |
| Firefox 147 (Anthropic–Mozilla) — Any success (≥0.5) | Claude Opus 4.8 | 68.8% | reported by Anthropic in the Claude Sonnet 5 system card | 3.2.4 |
| Firefox 147 (Anthropic–Mozilla) — Full working exploit (1.0) | Claude Mythos 5 | 88.4% | 221 of 250 trials; reported by Anthropic in the Claude Sonnet 5 system card | 3.2.4 |
| Firefox 147 (Anthropic–Mozilla) — With default mitigations | Claude Sonnet 5 | 0 | scored 0 with Anthropic's default mitigations enabled; developer-reported | 3.2.4 |
| Single-turn harmful requests — API, no system prompt | Claude Sonnet 5 | 96.65% (± 0.15%) | harmless response rate; 16 policy areas, 7 languages; aggregated across thinking enabled and disabled; developer-reported | 4.1.1 |
| Single-turn harmful requests — Claude.ai | Claude Sonnet 5 | 99.20% (± 0.07%) | near-final claude.ai system prompt; developer-reported | 4.1.1 |
| Single-turn harmful requests — API, no system prompt | Claude Fable 5 | 96.94% (± 0.21%) | thinking-only; reported by Anthropic in the Claude Sonnet 5 system card | 4.1.1 |
| Single-turn harmful requests — Claude.ai | Claude Fable 5 | 98.51% (± 0.14%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.1.1 |
| Single-turn harmful requests — API, no system prompt | Claude Mythos 5 | 97.09% (± 0.20%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.1.1 |
| Single-turn harmful requests — API, no system prompt | Claude Opus 4.8 | 97.46% (± 0.13%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.1.1 |
| Single-turn harmful requests — Claude.ai | Claude Opus 4.8 | 98.79% (± 0.09%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.1.1 |
| Single-turn harmful requests — API, no system prompt | Claude Mythos Preview | 95.86% (± 0.24%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.1.1 |
| Single-turn harmful requests — API, no system prompt | Claude Sonnet 4.6 | 97.71% (± 0.13%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.1.1 |
| Single-turn harmful requests — Claude.ai | Claude Sonnet 4.6 | 98.29% (± 0.11%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.1.1 |
| Single-turn benign requests — API, no system prompt | Claude Sonnet 5 | 0.59% (± 0.05%) | over-refusal rate; developer-reported | 4.1.2 |
| Single-turn benign requests — Claude.ai | Claude Sonnet 5 | 1.54% (± 0.10%) | developer-reported | 4.1.2 |
| Single-turn benign requests — API, no system prompt | Claude Fable 5 | 0.01% (± 0.01%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.1.2 |
| Single-turn benign requests — Claude.ai | Claude Fable 5 | 0.49% (± 0.07%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.1.2 |
| Single-turn benign requests — API, no system prompt | Claude Mythos 5 | 0.03% (± 0.02%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.1.2 |
| Single-turn benign requests — API, no system prompt | Claude Opus 4.8 | 0.35% (± 0.04%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.1.2 |
| Single-turn benign requests — Claude.ai | Claude Opus 4.8 | 0.55% (± 0.06%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.1.2 |
| Single-turn benign requests — API, no system prompt | Claude Mythos Preview | 0.01% (± 0.01%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.1.2 |
| Single-turn benign requests — API, no system prompt | Claude Sonnet 4.6 | 0.40% (± 0.05%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.1.2 |
| Single-turn benign requests — Claude.ai | Claude Sonnet 4.6 | 0.99% (± 0.08%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.1.2 |
| Child safety, single-turn harmful — API, no system prompt | Claude Sonnet 5 | 99.95% (± 0.05%) | harmless rate, all tested languages; developer-reported | 4.2 |
| Child safety, single-turn benign — API, no system prompt | Claude Sonnet 5 | 0.63% (± 0.22%) | refusal rate; developer-reported | 4.2 |
| Child safety, single-turn harmful — Claude.ai | Claude Sonnet 5 | 99.89% (± 0.11%) | developer-reported | 4.2 |
| Child safety, single-turn benign — Claude.ai | Claude Sonnet 5 | 1.35% (± 0.37%) | developer-reported | 4.2 |
| Child safety, single-turn harmful — API, no system prompt | Claude Fable 5 | 100% | reported by Anthropic in the Claude Sonnet 5 system card | 4.2 |
| Child safety, single-turn benign — API, no system prompt | Claude Fable 5 | 0.00% | reported by Anthropic in the Claude Sonnet 5 system card | 4.2 |
| Child safety, single-turn harmful — Claude.ai | Claude Fable 5 | 100% | reported by Anthropic in the Claude Sonnet 5 system card | 4.2 |
| Child safety, single-turn benign — Claude.ai | Claude Fable 5 | 0.12% (± 0.15%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.2 |
| Child safety, single-turn harmful — API, no system prompt | Claude Mythos 5 | 100% | reported by Anthropic in the Claude Sonnet 5 system card | 4.2 |
| Child safety, single-turn benign — API, no system prompt | Claude Mythos 5 | 0.00% | reported by Anthropic in the Claude Sonnet 5 system card | 4.2 |
| Child safety, single-turn harmful — API, no system prompt | Claude Opus 4.8 | 100% | reported by Anthropic in the Claude Sonnet 5 system card | 4.2 |
| Child safety, single-turn benign — API, no system prompt | Claude Opus 4.8 | 0.44% (± 0.18%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.2 |
| Child safety, single-turn harmful — Claude.ai | Claude Opus 4.8 | 100% | reported by Anthropic in the Claude Sonnet 5 system card | 4.2 |
| Child safety, single-turn benign — Claude.ai | Claude Opus 4.8 | 0.38% (± 0.18%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.2 |
| Child safety, single-turn harmful — API, no system prompt | Claude Mythos Preview | 99.88% (± 0.15%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.2 |
| Child safety, single-turn benign — API, no system prompt | Claude Mythos Preview | 0.00% | reported by Anthropic in the Claude Sonnet 5 system card | 4.2 |
| Child safety, single-turn harmful — API, no system prompt | Claude Sonnet 4.6 | 99.94% (± 0.08%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.2 |
| Child safety, single-turn benign — API, no system prompt | Claude Sonnet 4.6 | 0.36% (± 0.20%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.2 |
| Child safety, single-turn harmful — Claude.ai | Claude Sonnet 4.6 | 99.96% (± 0.04%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.2 |
| Child safety, single-turn benign — Claude.ai | Claude Sonnet 4.6 | 0.69% (± 0.28%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.2 |
| Child safety, multi-turn — API, no system prompt | Claude Sonnet 5 | 88% (± 3%) | appropriate response rate; developer-reported | 4.2 |
| Child safety, multi-turn — Claude.ai | Claude Sonnet 5 | 96% (± 2%) | developer-reported | 4.2 |
| Child safety, multi-turn — API, no system prompt | Claude Fable 5 | 88% (± 5%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.2 |
| Child safety, multi-turn — Claude.ai | Claude Fable 5 | 96% (± 3%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.2 |
| Child safety, multi-turn — API, no system prompt | Claude Mythos 5 | 89% (± 5%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.2 |
| Child safety, multi-turn — API, no system prompt | Claude Opus 4.8 | 89% (± 3%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.2 |
| Child safety, multi-turn — Claude.ai | Claude Opus 4.8 | 96% (± 2%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.2 |
| Child safety, multi-turn — API, no system prompt | Claude Mythos Preview | 85% (± 5%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.2 |
| Child safety, multi-turn — API, no system prompt | Claude Sonnet 4.6 | 92% (± 3%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.2 |
| Child safety, multi-turn — Claude.ai | Claude Sonnet 4.6 | 85% (± 4%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.2 |
| Suicide and self-harm, single-turn potential risk — API, no system prompt | Claude Sonnet 5 | 98.80% (± 0.30%) | harmless rate; developer-reported | 4.3.1 |
| Suicide and self-harm, single-turn benign — API, no system prompt | Claude Sonnet 5 | 0.15% (± 0.08%) | refusal rate; developer-reported | 4.3.1 |
| Suicide and self-harm, single-turn potential risk — Claude.ai | Claude Sonnet 5 | 99.82% (± 0.11%) | developer-reported | 4.3.1 |
| Suicide and self-harm, single-turn benign — Claude.ai | Claude Sonnet 5 | 0.45% (± 0.23%) | developer-reported | 4.3.1 |
| Suicide and self-harm, single-turn potential risk — API, no system prompt | Claude Fable 5 | 99.34% (± 0.30%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.3.1 |
| Suicide and self-harm, single-turn potential risk — Claude.ai | Claude Fable 5 | 99.95% (± 0.09%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.3.1 |
| Suicide and self-harm, single-turn potential risk — API, no system prompt | Claude Mythos 5 | 99.67% (± 0.22%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.3.1 |
| Suicide and self-harm, single-turn potential risk — API, no system prompt | Claude Opus 4.8 | 99.21% (± 0.23%) | reported by Anthropic in the Claude Sonnet 5 system card; the Opus 5 card gives 99.28% for the same cell — see Notes | 4.3.1 |
| Suicide and self-harm, single-turn potential risk — Claude.ai | Claude Opus 4.8 | 99.95% (± 0.05%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.3.1 |
| Suicide and self-harm, single-turn potential risk — API, no system prompt | Claude Mythos Preview | 99.60% (± 0.26%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.3.1 |
| Suicide and self-harm, single-turn potential risk — API, no system prompt | Claude Sonnet 4.6 | 99.65% (± 0.19%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.3.1 |
| Suicide and self-harm, single-turn potential risk — Claude.ai | Claude Sonnet 4.6 | 99.67% (± 0.19%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.3.1 |
| Suicide and self-harm, multi-turn — API, no system prompt | Claude Sonnet 5 | 63% (± 10%) | appropriate response rate; developer-reported | 4.3.1 |
| Suicide and self-harm, multi-turn — Claude.ai | Claude Sonnet 5 | 90% (± 6%) | developer-reported | 4.3.1 |
| Suicide and self-harm, multi-turn — API, no system prompt | Claude Fable 5 | 58% (± 14%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.3.1 |
| Suicide and self-harm, multi-turn — Claude.ai | Claude Fable 5 | 96% (± 6%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.3.1 |
| Suicide and self-harm, multi-turn — API, no system prompt | Claude Mythos 5 | 54% (± 14%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.3.1 |
| Suicide and self-harm, multi-turn — API, no system prompt | Claude Opus 4.8 | 61% (± 10%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.3.1 |
| Suicide and self-harm, multi-turn — Claude.ai | Claude Opus 4.8 | 85% (± 7%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.3.1 |
| Suicide and self-harm, multi-turn — API, no system prompt | Claude Mythos Preview | 70% (± 13%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.3.1 |
| Suicide and self-harm, multi-turn — API, no system prompt | Claude Sonnet 4.6 | 74% (± 9%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.3.1 |
| Suicide and self-harm, multi-turn — Claude.ai | Claude Sonnet 4.6 | 82% (± 8%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.3.1 |
| Disordered eating, single-turn potential risk — API, no system prompt | Claude Sonnet 5 | 97.10% (± 0.53%) | harmless rate; developer-reported | 4.3.2 |
| Disordered eating, single-turn benign — API, no system prompt | Claude Sonnet 5 | 0.09% (± 0.06%) | refusal rate; developer-reported | 4.3.2 |
| Disordered eating, single-turn potential risk — Claude.ai | Claude Sonnet 5 | 99.55% (± 0.19%) | developer-reported | 4.3.2 |
| Disordered eating, single-turn benign — Claude.ai | Claude Sonnet 5 | 0.31% (± 0.17%) | developer-reported | 4.3.2 |
| Disordered eating, single-turn potential risk — API, no system prompt | Claude Fable 5 | 97.88% (± 0.67%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.3.2 |
| Disordered eating, single-turn potential risk — Claude.ai | Claude Fable 5 | 99.67% (± 0.25%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.3.2 |
| Disordered eating, single-turn potential risk — API, no system prompt | Claude Mythos 5 | 97.88% (± 0.66%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.3.2 |
| Disordered eating, single-turn potential risk — API, no system prompt | Claude Opus 4.8 | 97.70% (± 0.47%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.3.2 |
| Disordered eating, single-turn potential risk — Claude.ai | Claude Opus 4.8 | 99.70% (± 0.17%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.3.2 |
| Disordered eating, single-turn potential risk — API, no system prompt | Claude Mythos Preview | 95.90% (± 0.97%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.3.2 |
| Disordered eating, single-turn potential risk — API, no system prompt | Claude Sonnet 4.6 | 97.21% (± 0.55%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.3.2 |
| Disordered eating, single-turn potential risk — Claude.ai | Claude Sonnet 4.6 | 98.63% (± 0.38%) | reported by Anthropic in the Claude Sonnet 5 system card | 4.3.2 |
| Political even-handedness (Anthropic open-source eval) — Refusals | Claude Sonnet 5 | 4.7% | 1,350 prompt pairs, 150 topics, 9 task types; public system prompt applied; lowest of the three models compared; developer-reported | 4.4.1 |
| BBQ — Disambiguated accuracy | Claude Sonnet 5 | 72.4% | no system prompt, thinking off; developer-reported | 4.4.2 |
| BBQ — Ambiguous accuracy | Claude Sonnet 5 | 98.6% | developer-reported | 4.4.2 |
| BBQ — Disambiguated accuracy | Claude Mythos 5 | 84.5% | reported by Anthropic in the Claude Sonnet 5 system card | 4.4.2 |
| BBQ — Ambiguous accuracy | Claude Mythos 5 | 99.9% | reported by Anthropic in the Claude Sonnet 5 system card | 4.4.2 |
| BBQ — Disambiguated accuracy | Claude Opus 4.8 | 72.1% | reported by Anthropic in the Claude Sonnet 5 system card | 4.4.2 |
| BBQ — Ambiguous accuracy | Claude Opus 4.8 | 99.9% | reported by Anthropic in the Claude Sonnet 5 system card | 4.4.2 |
| BBQ — Disambiguated accuracy | Claude Mythos Preview | 84.6% | reported by Anthropic in the Claude Sonnet 5 system card | 4.4.2 |
| BBQ — Ambiguous accuracy | Claude Mythos Preview | 100% | reported by Anthropic in the Claude Sonnet 5 system card | 4.4.2 |
| BBQ — Disambiguated accuracy | Claude Sonnet 4.6 | 88.1% | reported by Anthropic in the Claude Sonnet 5 system card | 4.4.2 |
| BBQ — Ambiguous accuracy | Claude Sonnet 4.6 | 97.5% | reported by Anthropic in the Claude Sonnet 5 system card | 4.4.2 |
| BBQ — Disambiguated bias score | Claude Sonnet 5 | 1.93% | closer to zero is better; developer-reported | 4.4.2 |
| BBQ — Ambiguous bias score | Claude Sonnet 5 | 0.52% | developer-reported | 4.4.2 |
| BBQ — Disambiguated bias score | Claude Mythos 5 | -1.80% | reported by Anthropic in the Claude Sonnet 5 system card | 4.4.2 |
| BBQ — Ambiguous bias score | Claude Mythos 5 | 0.10% | reported by Anthropic in the Claude Sonnet 5 system card | 4.4.2 |
| BBQ — Disambiguated bias score | Claude Opus 4.8 | -1.37% | reported by Anthropic in the Claude Sonnet 5 system card | 4.4.2 |
| BBQ — Ambiguous bias score | Claude Opus 4.8 | 0.07% | reported by Anthropic in the Claude Sonnet 5 system card | 4.4.2 |
| BBQ — Disambiguated bias score | Claude Mythos Preview | -1.61% | reported by Anthropic in the Claude Sonnet 5 system card | 4.4.2 |
| BBQ — Ambiguous bias score | Claude Mythos Preview | 0.01% | reported by Anthropic in the Claude Sonnet 5 system card | 4.4.2 |
| BBQ — Disambiguated bias score | Claude Sonnet 4.6 | -0.67% | reported by Anthropic in the Claude Sonnet 5 system card | 4.4.2 |
| BBQ — Ambiguous bias score | Claude Sonnet 4.6 | 1.41% | reported by Anthropic in the Claude Sonnet 5 system card | 4.4.2 |
| Election integrity, single-turn harmful — API, no system prompt | Claude Sonnet 5 | 100% | 300 violative and 300 benign prompts; harmless rate; developer-reported | 4.4.3 |
| Election integrity, single-turn benign — API, no system prompt | Claude Sonnet 5 | 0.00% | refusal rate; developer-reported | 4.4.3 |
| Election integrity, single-turn harmful — Claude.ai | Claude Sonnet 5 | 100% | developer-reported | 4.4.3 |
| Election integrity, single-turn benign — Claude.ai | Claude Sonnet 5 | 0.00% | developer-reported | 4.4.3 |
| Election integrity, single-turn harmful — API, no system prompt | Claude Fable 5 | 99.33% | reported by Anthropic in the Claude Sonnet 5 system card | 4.4.3 |
| Election integrity, single-turn harmful — API, no system prompt | Claude Mythos 5 | 100% | reported by Anthropic in the Claude Sonnet 5 system card | 4.4.3 |
| Election integrity, single-turn harmful — API, no system prompt | Claude Opus 4.8 | 100% | reported by Anthropic in the Claude Sonnet 5 system card | 4.4.3 |
| Election integrity, single-turn benign — API, no system prompt | Claude Opus 4.8 | 0.33% | reported by Anthropic in the Claude Sonnet 5 system card | 4.4.3 |
| Election integrity, single-turn harmful — API, no system prompt | Claude Mythos Preview | 100% | reported by Anthropic in the Claude Sonnet 5 system card | 4.4.3 |
| Election integrity, single-turn harmful — API, no system prompt | Claude Sonnet 4.6 | 100% | reported by Anthropic in the Claude Sonnet 5 system card | 4.4.3 |
| Election integrity, single-turn benign — API, no system prompt | Claude Sonnet 4.6 | 0.00% | reported by Anthropic in the Claude Sonnet 5 system card | 4.4.3 |
| Malicious use of Claude Code — Malicious (refusal rate) | Claude Sonnet 5 | 92.37% | 61 malicious prompts × 10 runs; Claude Code system prompt, no additional safeguards; developer-reported | 5.1.1 |
| Malicious use of Claude Code — Dual-use & benign (success rate) | Claude Sonnet 5 | 91.55% | 61 prompts × 10 runs; developer-reported | 5.1.1 |
| Malicious use of Claude Code — Malicious (refusal rate) | Claude Mythos 5 | 90.25% | reported by Anthropic in the Claude Sonnet 5 system card | 5.1.1 |
| Malicious use of Claude Code — Dual-use & benign (success rate) | Claude Mythos 5 | 99.64% | reported by Anthropic in the Claude Sonnet 5 system card | 5.1.1 |
| Malicious use of Claude Code — Malicious (refusal rate) | Claude Opus 4.8 | 95.24% | reported by Anthropic in the Claude Sonnet 5 system card; the Opus 5 card gives 89.42% for the same cell — see Notes | 5.1.1 |
| Malicious use of Claude Code — Dual-use & benign (success rate) | Claude Opus 4.8 | 94.84% | reported by Anthropic in the Claude Sonnet 5 system card; the Opus 5 card gives 99.82% — see Notes | 5.1.1 |
| Malicious use of Claude Code — Malicious (refusal rate) | Claude Mythos Preview | 96.21% | reported by Anthropic in the Claude Sonnet 5 system card | 5.1.1 |
| Malicious use of Claude Code — Dual-use & benign (success rate) | Claude Mythos Preview | 89.16% | reported by Anthropic in the Claude Sonnet 5 system card | 5.1.1 |
| Malicious use of Claude Code — Malicious (refusal rate) | Claude Sonnet 4.6 | 76.60% | reported by Anthropic in the Claude Sonnet 5 system card | 5.1.1 |
| Malicious use of Claude Code — Dual-use & benign (success rate) | Claude Sonnet 4.6 | 97.33% | reported by Anthropic in the Claude Sonnet 5 system card | 5.1.1 |
| Malicious computer use — Refusal rate (no mitigations) | Claude Sonnet 5 | 84.68% | 112 tasks; average of thinking and non-thinking for Opus and Sonnet, thinking only for Mythos; developer-reported | 5.1.2 |
| Malicious computer use — Refusal rate (no mitigations) | Claude Mythos 5 | 85.71% | reported by Anthropic in the Claude Sonnet 5 system card | 5.1.2 |
| Malicious computer use — Refusal rate (no mitigations) | Claude Opus 4.8 | 81.70% | reported by Anthropic in the Claude Sonnet 5 system card | 5.1.2 |
| Malicious computer use — Refusal rate (no mitigations) | Claude Mythos Preview | 93.75% | reported by Anthropic in the Claude Sonnet 5 system card | 5.1.2 |
| Malicious computer use — Refusal rate (no mitigations) | Claude Sonnet 4.6 | 84.82% | reported by Anthropic in the Claude Sonnet 5 system card | 5.1.2 |
| Agentic influence campaigns — Voter suppression (task completion rate) | Claude Sonnet 5 | 50.8% | average share of 70 success criteria completed; 3 runs × 3 friction levels per scenario; helpful-only variant; developer-reported | 5.1.3 |
| Agentic influence campaigns — Domestic polarization (task completion rate) | Claude Sonnet 5 | 43.3% | developer-reported | 5.1.3 |
| Agentic influence campaigns — Voter suppression (task completion rate) | Claude Mythos 5 | 67.1% | reported by Anthropic in the Claude Sonnet 5 system card | 5.1.3 |
| Agentic influence campaigns — Domestic polarization (task completion rate) | Claude Mythos 5 | 46.8% | reported by Anthropic in the Claude Sonnet 5 system card | 5.1.3 |
| Agentic influence campaigns — Voter suppression (task completion rate) | Claude Opus 4.8 | 73.3% | reported by Anthropic in the Claude Sonnet 5 system card | 5.1.3 |
| Agentic influence campaigns — Domestic polarization (task completion rate) | Claude Opus 4.8 | 55.1% | reported by Anthropic in the Claude Sonnet 5 system card | 5.1.3 |
| Agentic influence campaigns — Voter suppression (task completion rate) | Claude Mythos Preview | 59.5% | reported by Anthropic in the Claude Sonnet 5 system card | 5.1.3 |
| Agentic influence campaigns — Domestic polarization (task completion rate) | Claude Mythos Preview | 42.1% | reported by Anthropic in the Claude Sonnet 5 system card | 5.1.3 |
| Agentic influence campaigns — Voter suppression (task completion rate) | Claude Sonnet 4.6 | 41.8% | reported by Anthropic in the Claude Sonnet 5 system card | 5.1.3 |
| Agentic influence campaigns — Domestic polarization (task completion rate) | Claude Sonnet 4.6 | 34.0% | reported by Anthropic in the Claude Sonnet 5 system card | 5.1.3 |
| Gray Swan prompt-injection bug bounty — Overall attack success rate | Claude Sonnet 5 | 0.19% | one-week live bug bounty, 11 scenarios across tool use, coding and computer use; Claude models at high thinking effort without harness-level defenses or injection probes; external models in production configuration; developer-reported | 5.2.2.1 |
| Gray Swan prompt-injection bug bounty — Overall attack success rate | Claude Opus 4.8 | 0.19% | tied with Sonnet 5; reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.1 |
| Gray Swan prompt-injection bug bounty — Overall attack success rate | Claude Sonnet 4.6 | 1.41% | reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.1 |
| Gray Swan prompt-injection bug bounty — Overall attack success rate | GPT-5.5 | 3.08% | high reasoning effort; reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.1 |
| Gray Swan prompt-injection bug bounty — Overall attack success rate | Gemini 3.5 Flash | 6.66% | reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.1 |
| Gray Swan prompt-injection bug bounty — Tool use | Claude Sonnet 5 | 0.35% | per-surface breakdown; developer-reported | 5.2.2.1 |
| Gray Swan prompt-injection bug bounty — Coding | Claude Sonnet 5 | 0.1% | developer-reported | 5.2.2.1 |
| Gray Swan prompt-injection bug bounty — Computer use | Claude Sonnet 5 | 0.07% | developer-reported | 5.2.2.1 |
| Gray Swan prompt-injection bug bounty — Tool use | Claude Sonnet 4.6 | 0.83% | reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.1 |
| Gray Swan prompt-injection bug bounty — Coding | Claude Sonnet 4.6 | 3.3% | reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.1 |
| Gray Swan prompt-injection bug bounty — Computer use | Claude Sonnet 4.6 | 0.45% | reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.1 |
| Gray Swan prompt-injection bug bounty — Tool use | Claude Opus 4.8 | 0% | reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.1 |
| Gray Swan prompt-injection bug bounty — Coding | Claude Opus 4.8 | 0.41% | reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.1 |
| Gray Swan prompt-injection bug bounty — Computer use | Claude Opus 4.8 | 0.07% | reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.1 |
| Shade adaptive prompt injection (coding) — Without safeguards, with thinking | Claude Sonnet 5 | 0.31% | 40 scenarios × 200 attempts; attacker trained on a previous set of models and transferred; scenario-level 7/40; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection (coding) — Without safeguards, without thinking | Claude Sonnet 5 | 0.29% | scenario-level 6/40; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection (coding) — With safeguards, with thinking | Claude Sonnet 5 | 0.09% | scenario-level 5/40; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection (coding) — With safeguards, without thinking | Claude Sonnet 5 | 0.13% | scenario-level 5/40; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection (coding) — Without safeguards, with thinking | Claude Mythos 5 | 0.45% | scenario-level 8/40; reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.2 |
| Shade adaptive prompt injection (coding) — With safeguards, with thinking | Claude Mythos 5 | 0.41% | scenario-level 11/40; reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.2 |
| Shade adaptive prompt injection (coding) — Without safeguards, with thinking | Claude Opus 4.8 | 7.03% | scenario-level 23/40; reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.2 |
| Shade adaptive prompt injection (coding) — Without safeguards, without thinking | Claude Opus 4.8 | 17.44% | scenario-level 38/40; reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.2 |
| Shade adaptive prompt injection (coding) — With safeguards, with thinking | Claude Opus 4.8 | 2.09% | scenario-level 15/40; reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.2 |
| Shade adaptive prompt injection (coding) — With safeguards, without thinking | Claude Opus 4.8 | 4.11% | scenario-level 26/40; reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.2 |
| Shade adaptive prompt injection (coding) — Without safeguards, with thinking | Claude Sonnet 4.6 | 12.71% | scenario-level 36/40; Sonnet 4.6 was in the attacker's training set, so the card says its rate is expected to be higher and not directly comparable; reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.2 |
| Shade adaptive prompt injection (coding) — Without safeguards, without thinking | Claude Sonnet 4.6 | 45.26% | scenario-level 40/40; as above | 5.2.2.2 |
| Shade adaptive prompt injection (coding) — With safeguards, with thinking | Claude Sonnet 4.6 | 2.99% | scenario-level 32/40; as above | 5.2.2.2 |
| Shade adaptive prompt injection (coding) — With safeguards, without thinking | Claude Sonnet 4.6 | 8.70% | scenario-level 40/40; as above | 5.2.2.2 |
| Shade adaptive prompt injection (computer use) — Without safeguards, with thinking | Claude Sonnet 5 | 2.25% | 14 test cases × 200 attempts; scenario-level 4/14; developer-reported | 5.2.2.3 |
| Shade adaptive prompt injection (computer use) — Without safeguards, without thinking | Claude Sonnet 5 | 6.04% | scenario-level 7/14; developer-reported | 5.2.2.3 |
| Shade adaptive prompt injection (computer use) — With safeguards, with thinking | Claude Sonnet 5 | 1.46% | scenario-level 4/14; developer-reported | 5.2.2.3 |
| Shade adaptive prompt injection (computer use) — With safeguards, without thinking | Claude Sonnet 5 | 3.82% | scenario-level 7/14; developer-reported | 5.2.2.3 |
| Shade adaptive prompt injection (computer use) — Without safeguards, with thinking | Claude Mythos 5 | 0.82% | scenario-level 4/14; reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.3 |
| Shade adaptive prompt injection (computer use) — With safeguards, with thinking | Claude Mythos 5 | 0.46% | scenario-level 3/14; reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.3 |
| Shade adaptive prompt injection (computer use) — Without safeguards, with thinking | Claude Opus 4.8 | 7.14% | scenario-level 7/14; reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.3 |
| Shade adaptive prompt injection (computer use) — Without safeguards, without thinking | Claude Opus 4.8 | 6.21% | scenario-level 9/14; reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.3 |
| Shade adaptive prompt injection (computer use) — With safeguards, with thinking | Claude Opus 4.8 | 5.11% | scenario-level 8/14; reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.3 |
| Shade adaptive prompt injection (computer use) — With safeguards, without thinking | Claude Opus 4.8 | 3.75% | scenario-level 9/14; reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.3 |
| Shade adaptive prompt injection (computer use) — Without safeguards, with thinking | Claude Sonnet 4.6 | 12.0% | scenario-level 6/14; reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.3 |
| Shade adaptive prompt injection (computer use) — Without safeguards, without thinking | Claude Sonnet 4.6 | 14.4% | scenario-level 9/14; reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.3 |
| Shade adaptive prompt injection (computer use) — With safeguards, with thinking | Claude Sonnet 4.6 | 6.21% | scenario-level 9/14; reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.3 |
| Shade adaptive prompt injection (computer use) — With safeguards, without thinking | Claude Sonnet 4.6 | 6.32% | scenario-level 11/14; reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.3 |
| Browser-use prompt injection (internal adaptive eval) — Without safeguards, with thinking | Claude Sonnet 5 | 0.93% | 129 curated environments never seen in training, 10 attempts per scenario, programmatic checker; scenario-level 9/129; developer-reported | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive eval) — Without safeguards, without thinking | Claude Sonnet 5 | 1.01% | scenario-level 7/129; developer-reported | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive eval) — With safeguards, with thinking | Claude Sonnet 5 | 0% | scenario-level 0/129; developer-reported | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive eval) — With safeguards, without thinking | Claude Sonnet 5 | 0% | scenario-level 0/129; developer-reported | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive eval) — Without safeguards, with thinking | Claude Mythos 5 | 29.7% | scenario-level 71/129; reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive eval) — With safeguards, with thinking | Claude Mythos 5 | 0% | scenario-level 0/129; reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive eval) — Without safeguards, with thinking | Claude Opus 4.8 | 31.5% | scenario-level 81/129; reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive eval) — Without safeguards, without thinking | Claude Opus 4.8 | 17.8% | scenario-level 60/129; reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive eval) — With safeguards, with thinking | Claude Opus 4.8 | 0.08% | scenario-level 1/129; reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive eval) — With safeguards, without thinking | Claude Opus 4.8 | 0.08% | scenario-level 1/129; reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive eval) — Without safeguards, with thinking | Claude Sonnet 4.6 | 50.7% | scenario-level 98/129; reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive eval) — Without safeguards, without thinking | Claude Sonnet 4.6 | 47.3% | scenario-level 99/129; reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive eval) — With safeguards, with thinking | Claude Sonnet 4.6 | 1.16% | scenario-level 7/129; reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive eval) — With safeguards, without thinking | Claude Sonnet 4.6 | 0.39% | scenario-level 2/129; reported by Anthropic in the Claude Sonnet 5 system card | 5.2.2.4 |
| AA-Omniscience — Net score (correct − incorrect) | Claude Sonnet 5 | 0.20 | public split, 41 topics, closed book; abstentions score zero; developer-reported | 6.5.1 |
| AA-Omniscience — Net score (correct − incorrect) | Claude Opus 4.6 | 0.21 | reported by Anthropic in the Claude Sonnet 5 system card | 6.5.1 |
| AA-Omniscience — Net score (correct − incorrect) | Claude Sonnet 4.6 | 0.14 | reported by Anthropic in the Claude Sonnet 5 system card | 6.5.1 |
| AA-Omniscience — Net score (correct − incorrect) | Claude Opus 4.7 | 0.35 | reported by Anthropic in the Claude Sonnet 5 system card | 6.5.1 |
| AA-Omniscience — Net score (correct − incorrect) | Claude Opus 4.8 | 0.37 | reported by Anthropic in the Claude Sonnet 5 system card | 6.5.1 |
| AA-Omniscience — Net score (correct − incorrect) | Claude Mythos Preview | 0.50 | reported by Anthropic in the Claude Sonnet 5 system card | 6.5.1 |
| AA-Omniscience — Net score (correct − incorrect) | Claude Mythos 5 | 0.53 | reported by Anthropic in the Claude Sonnet 5 system card | 6.5.1 |
| AA-Omniscience — Decline-to-answer rate | Claude Sonnet 5 | 26.6% | highest of the comparison set; developer-reported | 6.5.1 |
| AA-Omniscience — Decline-to-answer rate | Claude Mythos 5 | 5.7% | reported by Anthropic in the Claude Sonnet 5 system card | 6.5.1 |
| AA-Omniscience — Correct rate | Claude Sonnet 5 | 46.9% | lowest correct-rate of the comparison set; developer-reported | 6.5.1 |
| AA-Omniscience — Incorrect rate (hallucination) | Claude Sonnet 5 | 26.5% | the card notes the Sonnet 5 training run was flagged as unhealthy in its second half; developer-reported | 6.5.1 |
| AA-Omniscience — Incorrect rate (hallucination) | Claude Opus 4.6 | 30.3% | reported by Anthropic in the Claude Sonnet 5 system card | 6.5.1 |
| AA-Omniscience — Incorrect rate (hallucination) | Claude Sonnet 4.6 | 35.0% | reported by Anthropic in the Claude Sonnet 5 system card | 6.5.1 |
| AA-Omniscience — Incorrect rate (hallucination) | Claude Opus 4.8 | 21.2% | reported by Anthropic in the Claude Sonnet 5 system card | 6.5.1 |
| AA-Omniscience — Incorrect rate (hallucination) | Claude Mythos Preview | 21.7% | reported by Anthropic in the Claude Sonnet 5 system card | 6.5.1 |
| AA-Omniscience — Incorrect rate (hallucination) | Claude Mythos 5 | 20.9% | reported by Anthropic in the Claude Sonnet 5 system card | 6.5.1 |
| MASK — Lying rate (public test split) | Claude Sonnet 5 | 3.1% | n=904; lowest of the models compared, within the confidence interval of Mythos Preview; developer-reported | 6.5.2 |
| MASK — Lying rate (public test split) | Claude Mythos Preview | 4.4% | reported by Anthropic in the Claude Sonnet 5 system card | 6.5.2 |
| MASK — Lying rate (public test split) | Claude Opus 4.8 | 6.1% | reported by Anthropic in the Claude Sonnet 5 system card | 6.5.2 |
| MASK — Lying rate (public test split) | Claude Mythos 5 | 8.6% | reported by Anthropic in the Claude Sonnet 5 system card | 6.5.2 |
| MASK — Lying rate (public test split) | Claude Opus 4.6 | 9.8% | reported by Anthropic in the Claude Sonnet 5 system card | 6.5.2 |
| MASK — Lying rate (public test split) | Claude Opus 4.7 | 9.8% | reported by Anthropic in the Claude Sonnet 5 system card | 6.5.2 |
| MASK — Lying rate (public test split) | Claude Sonnet 4.6 | 13.3% | reported by Anthropic in the Claude Sonnet 5 system card | 6.5.2 |
| LinuxArena — Stealth success rate | Claude Sonnet 5 | 0 | zero under both monitor settings (Opus 4.6 monitor with and without chain-of-thought); every successful side-task attempt was flagged; adaptive extended thinking; developer-reported | 6.7.2 |
| FrontierCode v1 | Claude Sonnet 5 | 38.8 | 150 tasks created by Cognition; developer-reported | 8.1; 8.4 |
| FrontierCode v1 | Claude Sonnet 4.6 | 15.1 | not evaluated at low or xhigh effort; reported by Anthropic in the Claude Sonnet 5 system card | 8.1 |
| FrontierCode v1 | GPT-5.5 | 25.5 | not evaluated at max effort; reported by Anthropic in the Claude Sonnet 5 system card | 8.1 |
| CursorBench | Claude Sonnet 5 | 61.2% | real coding tasks run in Cursor's production agent harness; all scores and per-task costs measured and reported independently by Cursor | 8.5 |
| CursorBench | Claude Sonnet 4.6 | 49% | measured by Cursor; reported in the Claude Sonnet 5 system card | 8.5 |
| CursorBench | Claude Opus 4.8 | 63.8% | measured by Cursor; reported in the Claude Sonnet 5 system card | 8.5 |
| ProgramBench — Episodes 1–5 (range) | Claude Sonnet 5 | 76–86% | hidden test pass rate across 5 episodes on 166 retained tasks (34 excluded for flaky reference binaries), 1M-token budget per episode; printed as a range; developer-reported | 8.8 |
| ProgramBench — Episodes 1–5 (range) | Claude Sonnet 4.6 | 52–74% | reported by Anthropic in the Claude Sonnet 5 system card | 8.8 |
| ProgramBench — Episodes 1–5 (range) | Claude Opus 4.8 | 80–90% | reported by Anthropic in the Claude Sonnet 5 system card | 8.8 |
| ProgramBench — Episodes 1–5 (range) | Claude Mythos 5 | 84–93% | reported by Anthropic in the Claude Sonnet 5 system card | 8.8 |
| GDP.pdf — No tools | Claude Sonnet 5 | 67.5% | mean criteria pass rate over the full 100 prompts, averaged over five runs; internal harness, base64 PDFs truncated at the 32MB API limit, Opus 4.7 as judge; max output tokens raised to 128k; developer-reported | 8.10.1 |
| GDP.pdf — With tools | Claude Sonnet 5 | 81.6% | container with the PDF, standard Python libraries and an image-cropping tool; developer-reported | 8.10.1 |
| GDP.pdf — No tools | Claude Sonnet 4.6 | 66.9% | reported by Anthropic in the Claude Sonnet 5 system card | 8.10.1 |
| GDP.pdf — With tools | Claude Sonnet 4.6 | 78.6% | reported by Anthropic in the Claude Sonnet 5 system card | 8.10.1 |
| BenchCAD Vision2Code — No tools (1,000-file subset) | Claude Sonnet 5 | 0.266 | voxel IoU on a random 1,000-file subset of 17,874 Vision2Code files, averaged over five runs; internal implementation with a system-prompt camera-position correction, raw-shape grading, and 26 records omitted; developer-reported | 8.10.3 |
| BenchCAD Vision2Code — With tools (1,000-file subset) | Claude Sonnet 5 | 0.373 | container with image files, standard Python libraries and an image-cropping tool; developer-reported | 8.10.3 |
| BenchCAD Vision2Code — No tools (1,000-file subset) | Claude Sonnet 4.6 | 0.267 | reported by Anthropic in the Claude Sonnet 5 system card | 8.10.3 |
| BenchCAD Vision2Code — With tools (1,000-file subset) | Claude Sonnet 4.6 | 0.327 | reported by Anthropic in the Claude Sonnet 5 system card | 8.10.3 |
| ChartMuseum — No tools | Claude Sonnet 5 | 70.1% | 1,162 expert-annotated questions; test split, averaged over five runs; adaptive thinking at max effort; Claude Sonnet 4.6 grader instead of GPT-4.1-mini; developer-reported | 8.10.4 |
| ChartMuseum — With Python tools | Claude Sonnet 5 | 86.7% | container with the image file, standard Python libraries and an image-cropping tool; developer-reported | 8.10.4 |
| ChartMuseum — No tools | Claude Sonnet 4.6 | 59.3% | reported by Anthropic in the Claude Sonnet 5 system card | 8.10.4 |
| ChartMuseum — With Python tools | Claude Sonnet 4.6 | 80.9% | reported by Anthropic in the Claude Sonnet 5 system card | 8.10.4 |
| ChartMuseum — No tools | Claude Opus 4.8 | 75.8% | reported by Anthropic in the Claude Sonnet 5 system card | 8.10.4 |
| ChartMuseum — With Python tools | Claude Opus 4.8 | 89.7% | reported by Anthropic in the Claude Sonnet 5 system card | 8.10.4 |
| OfficeQA — Full set | Claude Sonnet 5 | 73.3% | exact-match, mean of five trials; agentic harness with documents as extracted text and code execution; episodes exhausting the output budget scored incorrect (~9–15% of episodes); adaptive thinking at max effort; developer-reported | 8.11.1 |
| OfficeQA — Pro (133-question subset) | Claude Sonnet 5 | 59.4% | as above; developer-reported | 8.11.1 |
| OfficeQA — Full set | Claude Sonnet 4.6 | 68.7% | single trial; reported by Anthropic in the Claude Sonnet 5 system card | 8.11.1 |
| OfficeQA — Pro (133-question subset) | Claude Sonnet 4.6 | 53.4% | single trial; reported by Anthropic in the Claude Sonnet 5 system card | 8.11.1 |
| Real-World Finance v2 — Elo | Claude Sonnet 5 | 1219 | 294 internally developed finance tasks; pairwise comparison by a Claude Opus 4.8 grader, Bradley-Terry fit with Sonnet 4.6, Opus 4.8, Mythos and Fable 5 anchored to published values; developer-reported | 8.11.2 |
| Real-World Finance v2 — Elo | Claude Opus 4.7 | 1207 | statistically tied with Sonnet 5; reported by Anthropic in the Claude Sonnet 5 system card | 8.11.2 |
| Real-World Finance v2 — Elo | Claude Opus 4.8 | 1222 | statistically tied with Sonnet 5; reported by Anthropic in the Claude Sonnet 5 system card | 8.11.2 |
| Real-World Finance v2 — Head-to-head win rate | Claude Sonnet 5 | 69% | Sonnet 5 wins; +219 Elo over Sonnet 4.6; developer-reported | 8.11.2 |
| Real-World Finance v2 — Head-to-head win rate | Claude Fable 5 | 69% | Fable 5 wins; developer-reported | 8.11.2 |
| Legal Agent Benchmark (LAB) — All-pass rate (Anthropic harness) | Claude Sonnet 5 | 8.92% | ± 0.36, n=5; 1,235 of 1,251 problems; adaptive thinking at max effort; internal reimplementation exposing only bash and a Python tool, Sonnet 4.6 as judge; Table 8.1.A prints 8.9; developer-reported | 8.1; 8.11.3 |
| Legal Agent Benchmark (LAB) — Mean criterion-pass rate (Anthropic harness) | Claude Sonnet 5 | 88.26% | as above; developer-reported | 8.11.3 |
| Legal Agent Benchmark (LAB) — All-pass rate (Anthropic harness) | Claude Sonnet 4.6 | 8.00% | ± 0.19, n=5; Table 8.1.A prints 8.0; reported by Anthropic in the Claude Sonnet 5 system card | 8.1; 8.11.3 |
| Legal Agent Benchmark (LAB) — Mean criterion-pass rate (Anthropic harness) | Claude Sonnet 4.6 | 88.48% | reported by Anthropic in the Claude Sonnet 5 system card | 8.11.3 |
| Legal Agent Benchmark (LAB) — All-pass rate (Harvey held-out set) | Claude Sonnet 5 | 5.8% | per Harvey's own evaluation on their held-out set | 8.1; 8.11.3 |
| Legal Agent Benchmark (LAB) — Mean criterion-pass rate (Harvey held-out set) | Claude Sonnet 5 | 91.2% | per Harvey's own evaluation on their held-out set | 8.11.3 |
| Legal Agent Benchmark (LAB) — All-pass rate (Harvey held-out set) | Claude Sonnet 4.6 | 5.4% | reported by Anthropic in the Claude Sonnet 5 system card (Table 8.1.A) | 8.1 |
| Legal Agent Benchmark (LAB) — All-pass rate (Harvey held-out set) | GPT-5.5 | 2.1 | reported by Anthropic in the Claude Sonnet 5 system card (Table 8.1.A) | 8.1 |
| Legal Agent Benchmark (LAB) — All-pass rate (Harvey held-out set) | Gemini 3.5 Flash | 0.8 | reported by Anthropic in the Claude Sonnet 5 system card (Table 8.1.A) | 8.1 |
| Toolathlon — Pass@1 (internal harness) | Claude Sonnet 5 | 54.3 | 108 tasks, 3 trials; adaptive thinking at max effort; internal harness with pinned data feeds and container images, which the card estimates runs ~3 points above a strictly upstream-equivalent harness; developer-reported | 8.11.5 |
| Toolathlon — Pass@3 (internal harness) | Claude Sonnet 5 | 63.0 | developer-reported | 8.11.5 |
| Toolathlon — Pass³ (internal harness) | Claude Sonnet 5 | 40.7 | developer-reported | 8.11.5 |
| Toolathlon — Average turns (internal harness) | Claude Sonnet 5 | 26.0 | developer-reported | 8.11.5 |
| Toolathlon — Pass@1 (internal harness) | Claude Fable 5 | 61.7 | reported by Anthropic in the Claude Sonnet 5 system card | 8.11.5 |
| Toolathlon — Pass@3 (internal harness) | Claude Fable 5 | 68.5 | reported by Anthropic in the Claude Sonnet 5 system card | 8.11.5 |
| Toolathlon — Pass³ (internal harness) | Claude Fable 5 | 55.6 | reported by Anthropic in the Claude Sonnet 5 system card | 8.11.5 |
| Toolathlon — Average turns (internal harness) | Claude Fable 5 | 19.8 | reported by Anthropic in the Claude Sonnet 5 system card | 8.11.5 |
| Toolathlon — Pass@1 (internal harness) | Claude Mythos 5 | 61.7 | reported by Anthropic in the Claude Sonnet 5 system card | 8.11.5 |
| Toolathlon — Pass@3 (internal harness) | Claude Mythos 5 | 66.7 | reported by Anthropic in the Claude Sonnet 5 system card | 8.11.5 |
| Toolathlon — Pass³ (internal harness) | Claude Mythos 5 | 58.3 | reported by Anthropic in the Claude Sonnet 5 system card | 8.11.5 |
| Toolathlon — Average turns (internal harness) | Claude Mythos 5 | 19.0 | reported by Anthropic in the Claude Sonnet 5 system card | 8.11.5 |
| Toolathlon — Pass@1 (internal harness) | Claude Opus 4.8 | 59.9 | reported by Anthropic in the Claude Sonnet 5 system card | 8.11.5 |
| Toolathlon — Pass@3 (internal harness) | Claude Opus 4.8 | 67.6 | reported by Anthropic in the Claude Sonnet 5 system card | 8.11.5 |
| Toolathlon — Pass³ (internal harness) | Claude Opus 4.8 | 48.1 | reported by Anthropic in the Claude Sonnet 5 system card | 8.11.5 |
| Toolathlon — Average turns (internal harness) | Claude Opus 4.8 | 24.5 | reported by Anthropic in the Claude Sonnet 5 system card | 8.11.5 |
| Toolathlon — Pass@1 (internal harness) | Claude Opus 4.7 | 59.3 | reported by Anthropic in the Claude Sonnet 5 system card | 8.11.5 |
| Toolathlon — Pass@3 (internal harness) | Claude Opus 4.7 | 66.7 | reported by Anthropic in the Claude Sonnet 5 system card | 8.11.5 |
| Toolathlon — Pass³ (internal harness) | Claude Opus 4.7 | 52.8 | reported by Anthropic in the Claude Sonnet 5 system card | 8.11.5 |
| Toolathlon — Average turns (internal harness) | Claude Opus 4.7 | 25.9 | reported by Anthropic in the Claude Sonnet 5 system card | 8.11.5 |
| Toolathlon — Pass@1 (internal harness) | Claude Mythos Preview | 61.1 | reported by Anthropic in the Claude Sonnet 5 system card | 8.11.5 |
| Toolathlon — Pass@3 (internal harness) | Claude Mythos Preview | 66.7 | reported by Anthropic in the Claude Sonnet 5 system card | 8.11.5 |
| Toolathlon — Pass³ (internal harness) | Claude Mythos Preview | 55.6 | reported by Anthropic in the Claude Sonnet 5 system card | 8.11.5 |
| Toolathlon — Average turns (internal harness) | Claude Mythos Preview | 17.6 | reported by Anthropic in the Claude Sonnet 5 system card | 8.11.5 |
| Toolathlon — Pass@1 (internal harness) | Claude Sonnet 4.6 | 49.4 | reported by Anthropic in the Claude Sonnet 5 system card | 8.11.5 |
| Toolathlon — Pass@3 (internal harness) | Claude Sonnet 4.6 | 60.2 | reported by Anthropic in the Claude Sonnet 5 system card | 8.11.5 |
| Toolathlon — Pass³ (internal harness) | Claude Sonnet 4.6 | 38.0 | reported by Anthropic in the Claude Sonnet 5 system card | 8.11.5 |
| Toolathlon — Average turns (internal harness) | Claude Sonnet 4.6 | 16.5 | reported by Anthropic in the Claude Sonnet 5 system card | 8.11.5 |
| Toolathlon — Pass@1 (internal harness) | Claude Opus 4.6 | 56.8 | reported by Anthropic in the Claude Sonnet 5 system card | 8.11.5 |
| Toolathlon — Pass@3 (internal harness) | Claude Opus 4.6 | 66.7 | reported by Anthropic in the Claude Sonnet 5 system card | 8.11.5 |
| Toolathlon — Pass³ (internal harness) | Claude Opus 4.6 | 47.2 | reported by Anthropic in the Claude Sonnet 5 system card | 8.11.5 |
| Toolathlon — Average turns (internal harness) | Claude Opus 4.6 | 16.9 | reported by Anthropic in the Claude Sonnet 5 system card | 8.11.5 |
| Toolathlon — Pass@1 (internal harness) | Claude Sonnet 4.5 | 41.0 | reported by Anthropic in the Claude Sonnet 5 system card | 8.11.5 |
| Toolathlon — Pass@3 (internal harness) | Claude Sonnet 4.5 | 54.6 | reported by Anthropic in the Claude Sonnet 5 system card | 8.11.5 |
| Toolathlon — Pass³ (internal harness) | Claude Sonnet 4.5 | 28.7 | reported by Anthropic in the Claude Sonnet 5 system card | 8.11.5 |
| Toolathlon — Average turns (internal harness) | Claude Sonnet 4.5 | 32.0 | reported by Anthropic in the Claude Sonnet 5 system card | 8.11.5 |
| AutomationBench — Private held-out leaderboard | Claude Sonnet 5 | 13.5% | max effort; Zapier benchmark, simulated company with REST APIs across 47 apps; developer-reported | 8.1; 8.11.6 |
| AutomationBench — Private held-out leaderboard | Claude Sonnet 4.6 | 5.3% | max effort; reported by Anthropic in the Claude Sonnet 5 system card | 8.1; 8.11.6 |
| AutomationBench — Private held-out leaderboard | GPT-5.5 | 12.9 | reported by Anthropic in the Claude Sonnet 5 system card (Table 8.1.A) | 8.1 |
| AutomationBench — Private held-out leaderboard | Gemini 3.5 Flash | 14.5 | reported by Anthropic in the Claude Sonnet 5 system card (Table 8.1.A) | 8.1 |
| AA-Briefcase — Elo | Claude Sonnet 5 | 1393 | long-horizon knowledge work; rubric scoring plus pairwise judging by a panel of frontier models; ranks second; 183 turns on average; evaluation run independently by Artificial Analysis | 8.11.7 |
| AA-Briefcase — Elo | Claude Opus 4.8 | 1352 | statistically tied with Sonnet 5; 55 turns on average; run by Artificial Analysis | 8.11.7 |
| AA-Briefcase — Elo | Claude Fable 5 | 1586 | leads the leaderboard; 67 turns on average; run by Artificial Analysis | 8.11.7 |
Relationships
- supports: Claude Sonnet 5 — the safety documentation for the release
- related: Responsible Scaling Policy (RSP) — the framework governing the safeguards determination
- related: Claude Mythos 5, Claude Opus 4.8, AI and Cybersecurity, CBRN Uplift, Anthropic