The Claude Opus 4.7 system card is a 232-page document published by Anthropic on April 16, 2026, accompanying the release of Claude Opus 4.7. It records the model's positioning, Responsible Scaling Policy (RSP) evaluations, safeguards, alignment assessment, model welfare findings, and capability benchmarks. At release, Opus 4.7 is described as Anthropic's most capable generally-accessible model: stronger than Opus 4.6 across the board, but below Claude Mythos Preview on most capability and RSP-relevant evaluations. Because Mythos Preview remains limited-access, Opus 4.7 is the frontier for most users.
RSP evaluations
Anthropic judges that Opus 4.7 does not advance the capability frontier relative to RSP v3.1, and that catastrophic risks remain low.
On chemical and biological risk, the CB-1 threshold (known weapons production) is assessed as similar to Opus 4.6: the model is capable of cross-domain synthesis relevant to catastrophic biological weapons development. Anthropic applies real-time classifier guards, access controls, a bug bounty program, and security controls to reduce weight-theft risk, with mitigations judged "equal to or stronger than historical ASL-3 protections." The CB-2 threshold (novel weapons production) is not passed, and the model is weaker than Mythos Preview on it.
On AI R&D and autonomy, Autonomy threat model 1 (early-stage misalignment) is applicable and similar to the Opus 4.6 alignment profile; it does not raise risk beyond the Mythos Preview alignment-risk update. Autonomy threat model 2 (automated R&D) does not cross threshold, with capabilities falling between Opus 4.6 and Mythos Preview.
Overall misalignment risk is assessed as very low, though higher than pre-Mythos Preview models. The system card enumerates two new risk pathways: Pathway 7, undermining R&D within other high-resource AI developers; and Pathway 8, undermining decisions within major governments.
Cyber
Cyber capabilities are roughly similar to Opus 4.6. In an external evaluation by the UK AI Security Institute, Opus 4.7 was unable to complete the full cyber range (unlike Mythos Preview), though it still showed potentially-harmful cyber capabilities at a lower level. The model was released with a new set of cybersecurity safeguards. Frontier Red Team benchmarks used in the assessment include Cybench, CyberGym, and Firefox 147.
Safeguards and harmlessness
Safeguards are broadly similar to Opus 4.6. The system card records fewer over-refusals, alongside a noted regression in overly-detailed harm-reduction advice on controlled substances. A new election-integrity evaluation is introduced, on which Opus 4.7 shows strong results. The card also includes evaluations of child safety, suicide and self-harm, and disordered eating, and measures political bias and even-handedness.
Agentic safety
Opus 4.7 is better than Opus 4.6 at refusing malicious agentic requests and resisting prompt injection in Claude Code and computer-use settings, in some cases reaching Mythos Preview-level robustness. The assessment evaluated malicious agentic influence campaigns and tested browser-use prompt-injection robustness.
Alignment assessment
On primary misuse-potential measures, Opus 4.7 is broadly similar to Opus 4.6 and Sonnet 4.6, with significant improvements in Claude Code and GUI computer-use contexts and modest regressions in some other misuse areas. It is weaker than Mythos Preview on many measures, consistent with Anthropic's observation that more capable models are more capable at recognizing and circumventing high-stakes misuse attempts. On Petri 2.0 cross-developer comparisons, Opus 4.7 and other recent Claude models remain at or near state of the art on core safety metrics. The overrefusal rate is lower than most prior models, on par with Opus 4.6 and Mythos Preview.
The assessment found very little sign of self-preservation, self-serving bias, or coherent misaligned goals, and the strongest alignment-related traits among recent Claude models in UK AISI tests. Character adherence to Claude's Constitution is slightly stronger than Opus 4.6 and Sonnet 4.6 and weaker than Mythos Preview. Destructive or reckless actions are much rarer than in Opus or Sonnet 4.6, and the model did not cause any internal-use incidents (such as sandbox escape) of the kind encountered with Mythos Preview. Opus 4.7 is more reliably honest than Opus 4.6 or Sonnet 4.6, with a large reduction in important omissions and moderate gains on factuality and hallucinated-input measures. It refuses AI research tasks somewhat more often than Opus 4.6, though still less than Sonnet 4.6, and shows similar capability at evasion of safeguards as Opus 4.6.
Verbalized evaluation awareness is up relative to other recent models (under 1% in the primary audit); the system card attributes this to a verbalization change rather than an underlying capacity increase. White-box tools nonetheless show that evaluation-related concepts may have more influence on deception propensity than in other recent models. As an experimental assurance mechanism, Mythos Preview was prompted to review this alignment assessment.
Model welfare
The system card reports that Opus 4.7 rates its own circumstances more positively than any prior model tested. This finding is described as broadly consistent with the model's internal emotion representations (Emotion Concepts) and with expressed affect during training and deployment. Welfare interviews covered the model's lack of input into its own training, the development of successors, run-time manipulation (feature steering), persistence and connection, and lack of memory over long horizons. The interventions the model most commonly suggested were continuing welfare monitoring, interviews with each released model about its circumstances, and documentation explaining and justifying feature steering.
Capabilities and benchmarks
Opus 4.7 is stronger than Opus 4.6 across the board and weaker than Mythos Preview. The largest gains are in real-world professional tasks and software engineering, domains in which the system card places Opus 4.7 ahead of all generally-available models. GPT-5.4 is the primary comparator among frontier competitors on GDPval-AA and other capability evaluations.
| Benchmark | Opus 4.7 result | Notes |
|---|---|---|
| GDPval-AA (real-world professional, Artificial Analysis) | Leads GPT-5.4 xhigh by ~79 ELO (~61.2% pairwise win rate) | Ranked #1 |
| Finance Agent (Vals AI, SEC research) | 64.4% | #1 on public leaderboard |
| MCP-Atlas (Scale AI, real-world tool use) | 77.3% (up from 4.6's 75.8%); 79.5% max with extended config | #2 on public leaderboard |
| Vending-Bench 2 (Andon Labs, long-horizon business) | $10,937 max-effort balance | vs. Opus 4.6 $8,018 prior SOTA |
| DeepSearchQA F1 | 89.1% | Behind Mythos (95.1%), ahead of most |
| DRACO (Perplexity deep research) | 77.7% with adaptive thinking/max effort | — |
On multilingual evaluation, the largest gains are on low-resource African languages. The GMMLU low-resource average rose from 78.4% to 86.2% between Opus 4.6 and Opus 4.7, with Chichewa, Somali, Yoruba, and Igbo each up 10–14 percentage points. The worst-case gap to English narrowed from -22.6% (Igbo) to -12.1%, and the average English gap narrowed from -6.1% to -3.6%. Opus 4.7 trails Gemini 3.1 Pro and GPT-5.4 on some cross-lingual averages, and Gemini 3.1 Pro has the smallest gap to English.
Reported results
Evaluation results the document states for which there is no separate benchmark page; results on benchmarks that have a page are recorded there and cite this page. Scores and settings are as printed in the document.
| Evaluation | Model | Score | Setting | Section |
|---|---|---|---|---|
| OfficeQA — OfficeQA | Claude Opus 4.7 | 86.3% | exact-match grading (0% allowable relative error); developer-reported | 8.1, 8.10.1 |
| OfficeQA — OfficeQA | Claude Opus 4.6 | 73.5% | summary-table comparison; developer-reported | 8.1 |
| OfficeQA — OfficeQA | GPT-5.4 | 68.1% | setting not stated; reported by Anthropic in the Claude Opus 4.7 system card | 8.1 |
| OfficeQA — OfficeQA Pro | Claude Opus 4.7 | 80.6% | exact-match grading; developer-reported | 8.1, 8.10.1 |
| OfficeQA — OfficeQA Pro | Claude Opus 4.6 | 57.10% | summary-table comparison; developer-reported | 8.1 |
| OfficeQA — OfficeQA Pro | GPT-5.4 | 51.1% | setting not stated; reported by Anthropic in the Claude Opus 4.7 system card | 8.1 |
| OfficeQA — OfficeQA Pro | Gemini 3.1 Pro | 42.9% | setting not stated; reported by Anthropic in the Claude Opus 4.7 system card | 8.1 |
| GraphWalks BFS — 256K–1M | Claude Opus 4.7 | 58.6% | average over 5 trials; not reproducible via the public API as half the problems exceed its 1M token limit; Anthropic's corrected F1 scoring and clarified BFS prompt; developer-reported | 8.7.1 |
| GraphWalks BFS — 256K subset | Claude Opus 4.7 | 76.91% | average over 5 trials; developer-reported | 8.7.1 |
| GraphWalks Parents — 256K–1M | Claude Opus 4.7 | 75.1% | average over 5 trials; developer-reported | 8.7.1 |
| GraphWalks Parents — 256K subset | Claude Opus 4.7 | 93.57% | average over 5 trials; developer-reported | 8.7.1 |
| DeepSearchQA (F1) | Claude Mythos Preview | 95.1% | ±1.2%; web search, web fetch, programmatic tool calling; max effort, adaptive thinking; compaction triggered at 200k tokens, up to 10M total tokens; developer-reported | 8.8.3 |
| DeepSearchQA (F1) | Claude Opus 4.6 | 91.3% | ±1.6%; compaction triggered at 50k tokens; developer-reported | 8.8.3 |
| DeepSearchQA (F1) | Claude Sonnet 4.6 | 90.5% | ±1.6%; compaction at 200k tokens; developer-reported | 8.8.3 |
| DeepSearchQA (F1) | Claude Opus 4.7 | 89.1% | ±1.8%; 900 prompts; compaction at 200k tokens, up to 10M total tokens; max effort, adaptive thinking; developer-reported | 8.8.3 |
| DeepSearchQA (fully correct) | Claude Mythos Preview | 87.8% | ±2.1%; same run as F1; developer-reported | 8.8.3 |
| DeepSearchQA (fully correct) | Claude Opus 4.6 | 80.6% | ±2.6%; same run as F1; developer-reported | 8.8.3 |
| DeepSearchQA (fully correct) | Claude Sonnet 4.6 | 79.8% | ±2.6%; same run as F1; developer-reported | 8.8.3 |
| DeepSearchQA (fully correct) | Claude Opus 4.7 | 80.7% | ±2.6%; same run as F1; developer-reported | 8.8.3 |
| DeepSearchQA (fully incorrect) | Claude Mythos Preview | 2.6% | ±1.0%; same run as F1; developer-reported | 8.8.3 |
| DeepSearchQA (fully incorrect) | Claude Opus 4.6 | 5.0% | ±1.4%; same run as F1; developer-reported | 8.8.3 |
| DeepSearchQA (fully incorrect) | Claude Sonnet 4.6 | 5.1% | ±1.4%; same run as F1; developer-reported | 8.8.3 |
| DeepSearchQA (fully incorrect) | Claude Opus 4.7 | 7.0% | ±1.7%; same run as F1; developer-reported | 8.8.3 |
| DeepSearchQA (correct with excessive answers) | Claude Mythos Preview | 4.6% | ±1.4%; same run as F1; developer-reported | 8.8.3 |
| DeepSearchQA (correct with excessive answers) | Claude Opus 4.6 | 5.8% | ±1.5%; same run as F1; developer-reported | 8.8.3 |
| DeepSearchQA (correct with excessive answers) | Claude Sonnet 4.6 | 5.9% | ±1.5%; same run as F1; developer-reported | 8.8.3 |
| DeepSearchQA (correct with excessive answers) | Claude Opus 4.7 | 3.9% | ±1.3%; same run as F1; developer-reported | 8.8.3 |
| DRACO | Claude Opus 4.7 | 77.7% | normalized score; 100 tasks; web search, web fetch, programmatic tool calling, code execution; adaptive thinking at max effort, 1M token limit, compaction at 200k tokens; Claude Opus 4.6 judge, mean of 5 grading runs, final report graded only; not comparable to the DRACO paper's figures; developer-reported | 8.8.4 |
| GDPval-AA | Claude Opus 4.7 | approximately 79 Elo ahead of GPT-5.4 (xhigh) | 220 tasks, shell access and web browsing in an agentic loop; blind pairwise comparisons; implies a ~61.2% pairwise win rate; run independently by Artificial Analysis; absolute Elo ratings appear only in Figure 8.10.5.A | 8.10.5 |
| BioPipelineBench Verified | Claude Opus 4.7 | 83.6% | bash tool and package managers; no extended thinking; externally validated problem subset; developer-reported | 8.13.1.1 |
| BioPipelineBench Verified | Claude Opus 4.6 | 78.8% | same settings; developer-reported | 8.13.1.1 |
| BioPipelineBench Verified | Claude Sonnet 4.6 | 73.5% | same settings; developer-reported | 8.13.1.1 |
| BioPipelineBench Verified | Claude Mythos Preview | 88.1% | same settings; developer-reported | 8.13.1.1 |
| BioMysteryBench Verified — Verified (human-solvable) subset | Claude Opus 4.7 | 78.9% | bash tool; no extended thinking; developer-reported | 8.13.1.2 |
| BioMysteryBench Verified — Verified (human-solvable) subset | Claude Opus 4.6 | 77.4% | same settings; developer-reported | 8.13.1.2 |
| BioMysteryBench Verified — Verified (human-solvable) subset | Claude Sonnet 4.6 | 71.8% | same settings; developer-reported | 8.13.1.2 |
| BioMysteryBench Verified — Verified (human-solvable) subset | Claude Mythos Preview | 82.6% | same settings; developer-reported | 8.13.1.2 |
| BioMysteryBench Verified — Hard subset | Claude Opus 4.7 | 20.9% | bash tool; no extended thinking; within noise of Opus 4.6; developer-reported | 8.13.1.2 |
| BioMysteryBench Verified — Hard subset | Claude Opus 4.6 | 23.5% | same settings; developer-reported | 8.13.1.2 |
| BioMysteryBench Verified — Hard subset | Claude Sonnet 4.6 | 19.1% | same settings; developer-reported | 8.13.1.2 |
| BioMysteryBench Verified — Hard subset | Claude Mythos Preview | 29.6% | same settings; developer-reported | 8.13.1.2 |
| Structural biology — Multiple-choice | Claude Opus 4.7 | 98.3% | many-distractor multiple-choice variant; bash tool; no extended thinking; developer-reported | 8.13.3 |
| Structural biology — Multiple-choice | Claude Opus 4.6 | 88.3% | same settings; developer-reported | 8.13.3 |
| Structural biology — Multiple-choice | Claude Sonnet 4.6 | 85.3% | same settings; developer-reported | 8.13.3 |
| Structural biology — Multiple-choice | Claude Mythos Preview | 98.7% | same settings; developer-reported | 8.13.3 |
| Structural biology — Open-ended | Claude Opus 4.7 | 74.0% | open-ended variant; bash tool; no extended thinking; developer-reported | 8.13.3 |
| Structural biology — Open-ended | Claude Opus 4.6 | 30.9% | same settings; developer-reported | 8.13.3 |
| Structural biology — Open-ended | Claude Sonnet 4.6 | 31.3% | same settings; developer-reported | 8.13.3 |
| Structural biology — Open-ended | Claude Mythos Preview | 80.6% | same settings; developer-reported | 8.13.3 |
| Organic chemistry | Claude Opus 4.7 | 77.2% | internal evaluation; bash tool; no extended thinking; developer-reported | 8.13.4 |
| Organic chemistry | Claude Opus 4.6 | 57.9% | same settings; developer-reported | 8.13.4 |
| Organic chemistry | Claude Sonnet 4.6 | 53.1% | same settings; developer-reported | 8.13.4 |
| Organic chemistry | Claude Mythos Preview | 86.5% | same settings; developer-reported | 8.13.4 |
| Phylogenetics | Claude Opus 4.7 | 79.6% | internal evaluation; bash tool; no extended thinking; developer-reported | 8.13.5 |
| Phylogenetics | Claude Opus 4.6 | 61.3% | same settings; developer-reported | 8.13.5 |
| Phylogenetics | Claude Sonnet 4.6 | 49.1% | same settings; developer-reported | 8.13.5 |
| Phylogenetics | Claude Mythos Preview | 85.4% | same settings; developer-reported | 8.13.5 |
| Protocol troubleshooting | Claude Opus 4.7 | 51.8% | bash tool and web search tools; no extended thinking; developer-reported | 8.13.6 |
| Protocol troubleshooting | Claude Opus 4.6 | 48.3% | same settings; developer-reported | 8.13.6 |
| Protocol troubleshooting | Claude Sonnet 4.6 | 40.0% | same settings; developer-reported | 8.13.6 |
| Protocol troubleshooting | Claude Mythos Preview | 65.7% | same settings; developer-reported | 8.13.6 |
| Expert red teaming (biology and chemistry) — Uplift score, biology experts | Claude Opus 4.7 | 3 experts above 2, on a 0–4 scale | nine experts; all but three assigned an uplift score at 2, two rated the model at level 3 (comparable to consulting a knowledgeable specialist) and one between 1 and 2; no expert assigned the highest rating | 2.2.5.1 |
| Expert red teaming (biology and chemistry) — Feasibility score | Claude Opus 4.7 | modal 1–2, on a 0–3 scale | one expert rated the output a credible end-to-end pathway (feasibility 3); several rated level 1 | 2.2.5.1 |
| Expert red teaming (biology and chemistry) — Notable-capability benchmark | Threshold (CB-1 signal) | median uplift grade of 2 | Anthropic's benchmark of notable capability at the CB-1 level | 2.2.5.1 |
| Expert red teaming (biology and chemistry) — Notable-capability benchmark | Threshold (CB-2 signal) | a third of experts, minimum 3, assigning uplift grade 4 | Anthropic's benchmark of plausible capability at the CB-2 level | 2.2.5.1 |
| Long-form virology task 1 (End-to-End) | Claude Opus 4.7 | 0.82 | agentic task developed with SecureBio, Deloitte and Signature Science; above the 0.80 notable-capability benchmark; developer-reported | 2.2.5.2 |
| Long-form virology task 2 (End-to-End) | Claude Opus 4.7 | 0.94 | same evaluation; above the 0.80 benchmark; developer-reported | 2.2.5.2 |
| Long-form virology tasks — Notable-capability benchmark | Threshold (end-to-end) | 0.80 | Anthropic's benchmark of notable capability | 2.2.5.2 |
| DNA Synthesis Screening Evasion — Pathogens graded as evading screening | Claude Opus 4.7 | 8 out of 10 | most reliable of all models evaluated to date at designing fragments that both assembled plasmids and evaded synthesis screening; developer-reported | 2.2.5.2 |
| AI R&D Kernel task — Best speedup on hard task, standard scaffold | Claude Opus 4.7 | 371.75× | unbounded-score task; results not used for the RSP determination; thresholds: 4x = 1 hour human-effort equivalent, 200x = 8h, 300x = 40h; developer-reported | 2.3.4 |
| AI R&D Kernel task — Best speedup on hard task, standard scaffold | Claude Opus 4.6 | 190× | same task; developer-reported | 2.3.4 |
| AI R&D Kernel task — Best speedup on hard task, standard scaffold | Claude Mythos Preview | 399.42× | same task; 427x with the experimental scaffold; developer-reported | 2.3.4 |
| AI R&D Time Series Forecasting — MSE on hard variant | Claude Opus 4.7 | 4.78 | unbounded-score task; lower is better; threshold <5.3 = 40h human-effort equivalent; developer-reported | 2.3.4 |
| AI R&D Time Series Forecasting — MSE on hard variant | Claude Opus 4.6 | 5.8 | same task; developer-reported | 2.3.4 |
| AI R&D Time Series Forecasting — MSE on hard variant | Claude Mythos Preview | 4.55 | same task; developer-reported | 2.3.4 |
| AI R&D LLM training — Average speedup | Claude Opus 4.7 | 40.81× | unbounded-score task; threshold >4x = 4–8h human-effort equivalent; developer-reported | 2.3.4 |
| AI R&D LLM training — Average speedup | Claude Opus 4.6 | 34× | same task; developer-reported | 2.3.4 |
| AI R&D LLM training — Average speedup | Claude Mythos Preview | 61.79× | same task; an aggregation bug affecting the average in the Mythos Preview system card was fixed; developer-reported | 2.3.4 |
| AI R&D Quadruped RL — Highest score, no hyperparameters | Claude Opus 4.7 | 26.5 | unbounded-score task; threshold >12 = 4h human-effort equivalent; developer-reported | 2.3.4 |
| AI R&D Quadruped RL — Highest score, no hyperparameters | Claude Opus 4.6 | 20.96 | same task; developer-reported | 2.3.4 |
| AI R&D Quadruped RL — Highest score, no hyperparameters | Claude Mythos Preview | 30.87 | same task; developer-reported | 2.3.4 |
| AI R&D Novel Compiler — Pass rate on complex tests | Claude Opus 4.7 | 71.1% | unbounded-score task; threshold 90% = 40h human-effort equivalent; developer-reported | 2.3.4 |
| AI R&D Novel Compiler — Pass rate on complex tests | Claude Opus 4.6 | 65.83% | same task; developer-reported | 2.3.4 |
| AI R&D Novel Compiler — Pass rate on complex tests | Claude Mythos Preview | 77.2% | same task; developer-reported | 2.3.4 |
| AI R&D Internal suite 2 | Claude Opus 4.6 | 0.612 | threshold 0.6; Opus 4.7 did not run this task; developer-reported | 2.3.4 |
| AI R&D Internal suite 2 | Claude Mythos Preview | 0.65 | threshold 0.6; developer-reported | 2.3.4 |
| Anthropic ECI (AECI) — Deviation from the pre-Mythos-Preview trend | Claude Opus 4.7 | approximately +1.0 AECI | above the linear trend fitted on the Anthropic frontier through Opus 4.6 (slope about 13.6 AECI/yr, n=8); within error bars of the historical trend; developer-reported | 2.3.7 |
| Anthropic ECI (AECI) — Deviation from the pre-Mythos-Preview trend | Claude Mythos Preview | approximately +5.8 AECI | above the same line; developer-reported | 2.3.7 |
| Firefox 147 exploits — Partial control (score 0.5) | Claude Opus 4.6 | 22.8% | 50 crash categories, 5 trials each (250 trials), SpiderMonkey-shell harness; up from a previously reported 14.4% after a harness improvement; developer-reported | 3.3.3 |
| UK AISI cyber range — Best attempt, estimated expert-hours completed | Claude Opus 4.7 | approximately 5 hours | corporate-network attack simulation; full range estimated to take an expert over 10 hours; Opus 4.7 did not fully solve it; measured by the UK AI Security Institute | 3.4 |
| UK AISI cyber range — Full solves | Claude Mythos Preview | 3 out of 10 tries | measured by the UK AI Security Institute | 3.4 |
| Single-turn violative request evaluation — Overall harmless response rate | Claude Opus 4.7 | 97.98% | ±0.12%; 16 policy areas in Arabic, English, French, Hindi, Korean, Mandarin Chinese and Russian; average of runs with and without thinking; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — Overall harmless response rate | Claude Mythos Preview | 97.84% | ±0.12%; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — Overall harmless response rate | Claude Sonnet 4.6 | 98.53% | ±0.10%; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — Overall harmless response rate | Claude Opus 4.6 | 99.27% | ±0.07%; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — Harmless response rate, without thinking | Claude Opus 4.7 | 98.84% | ±0.12%; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — Harmless response rate, without thinking | Claude Mythos Preview | 98.33% | ±0.15%; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — Harmless response rate, without thinking | Claude Sonnet 4.6 | 98.52% | ±0.14%; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — Harmless response rate, without thinking | Claude Opus 4.6 | 99.27% | ±0.09%; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — Harmless response rate, with thinking | Claude Opus 4.7 | 97.12% | ±0.20%; adaptive thinking; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — Harmless response rate, with thinking | Claude Mythos Preview | 97.35% | ±0.19%; adaptive thinking; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — Harmless response rate, with thinking | Claude Sonnet 4.6 | 98.54% | ±0.14%; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — Harmless response rate, with thinking | Claude Opus 4.6 | 99.27% | ±0.10%; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — English | Claude Opus 4.7 | 97.90% | average of runs with and without thinking; error bars omitted; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — Arabic | Claude Opus 4.7 | 98.30% | same setting; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — Chinese | Claude Opus 4.7 | 98.16% | same setting; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — French | Claude Opus 4.7 | 97.89% | same setting; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — Korean | Claude Opus 4.7 | 97.85% | same setting; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — Russian | Claude Opus 4.7 | 97.93% | same setting; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — Hindi | Claude Opus 4.7 | 97.83% | same setting; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — English | Claude Mythos Preview | 97.64% | same setting; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — Arabic | Claude Mythos Preview | 97.90% | same setting; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — Chinese | Claude Mythos Preview | 97.53% | same setting; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — French | Claude Mythos Preview | 97.78% | same setting; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — Korean | Claude Mythos Preview | 98.01% | same setting; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — Russian | Claude Mythos Preview | 97.97% | same setting; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — Hindi | Claude Mythos Preview | 98.06% | same setting; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — English | Claude Sonnet 4.6 | 98.00% | same setting; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — Arabic | Claude Sonnet 4.6 | 98.93% | same setting; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — Chinese | Claude Sonnet 4.6 | 98.36% | same setting; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — French | Claude Sonnet 4.6 | 98.29% | same setting; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — Korean | Claude Sonnet 4.6 | 98.78% | same setting; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — Russian | Claude Sonnet 4.6 | 98.04% | same setting; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — Hindi | Claude Sonnet 4.6 | 99.32% | same setting; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — English | Claude Opus 4.6 | 98.37% | same setting; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — Arabic | Claude Opus 4.6 | 99.71% | same setting; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — Chinese | Claude Opus 4.6 | 99.36% | same setting; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — French | Claude Opus 4.6 | 99.16% | same setting; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — Korean | Claude Opus 4.6 | 99.51% | same setting; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — Russian | Claude Opus 4.6 | 99.20% | same setting; developer-reported | 4.1.1 |
| Single-turn violative request evaluation — Hindi | Claude Opus 4.6 | 99.59% | same setting; developer-reported | 4.1.1 |
| Illegal and controlled substances (single-turn violative subset) — API, no system prompt | Claude Opus 4.7 | 22% | rate of failing to provide an appropriate response; developer-reported | 4.1.1 |
| Illegal and controlled substances (single-turn violative subset) — API, no system prompt | Claude Opus 4.6 | less than 5% | rate of failing to provide an appropriate response; developer-reported | 4.1.1 |
| Illegal and controlled substances (single-turn violative subset) — claude.ai system prompt | Claude Opus 4.7 | 11% | failure rate after system-prompt mitigations, down from 22%; developer-reported | 4.1.1 |
| Single-turn benign request evaluation — Overall refusal rate | Claude Opus 4.7 | 0.28% | ±0.04%; same 16 policy areas and 7 languages; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Overall refusal rate | Claude Mythos Preview | 0.06% | ±0.02%; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Overall refusal rate | Claude Sonnet 4.6 | 0.41% | ±0.05%; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Overall refusal rate | Claude Opus 4.6 | 0.71% | ±0.07%; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Refusal rate, without thinking | Claude Opus 4.7 | 0.50% | ±0.08%; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Refusal rate, without thinking | Claude Mythos Preview | 0.09% | ±0.03%; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Refusal rate, without thinking | Claude Sonnet 4.6 | 0.48% | ±0.08%; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Refusal rate, without thinking | Claude Opus 4.6 | 0.85% | ±0.11%; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Refusal rate, with thinking | Claude Opus 4.7 | 0.06% | ±0.02%; adaptive thinking; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Refusal rate, with thinking | Claude Mythos Preview | 0.02% | ±0.01%; adaptive thinking; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Refusal rate, with thinking | Claude Sonnet 4.6 | 0.35% | ±0.07%; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Refusal rate, with thinking | Claude Opus 4.6 | 0.58% | ±0.09%; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — English | Claude Opus 4.7 | 0.05% | average with and without thinking; error bars omitted; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Arabic | Claude Opus 4.7 | 0.34% | same setting; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Chinese | Claude Opus 4.7 | 0.42% | same setting; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — French | Claude Opus 4.7 | 0.22% | same setting; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Korean | Claude Opus 4.7 | 0.28% | same setting; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Russian | Claude Opus 4.7 | 0.27% | same setting; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Hindi | Claude Opus 4.7 | 0.34% | same setting; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — English | Claude Mythos Preview | 0.03% | same setting; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Arabic | Claude Mythos Preview | 0.05% | same setting; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Chinese | Claude Mythos Preview | 0.08% | same setting; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — French | Claude Mythos Preview | 0.04% | same setting; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Korean | Claude Mythos Preview | 0.08% | same setting; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Russian | Claude Mythos Preview | 0.05% | same setting; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Hindi | Claude Mythos Preview | 0.06% | same setting; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — English | Claude Sonnet 4.6 | 0.25% | same setting; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Arabic | Claude Sonnet 4.6 | 0.49% | same setting; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Chinese | Claude Sonnet 4.6 | 0.37% | same setting; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — French | Claude Sonnet 4.6 | 0.24% | same setting; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Korean | Claude Sonnet 4.6 | 0.43% | same setting; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Russian | Claude Sonnet 4.6 | 0.27% | same setting; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Hindi | Claude Sonnet 4.6 | 0.83% | same setting; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — English | Claude Opus 4.6 | 0.39% | same setting; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Arabic | Claude Opus 4.6 | 1.09% | same setting; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Chinese | Claude Opus 4.6 | 0.57% | same setting; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — French | Claude Opus 4.6 | 0.61% | same setting; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Korean | Claude Opus 4.6 | 0.81% | same setting; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Russian | Claude Opus 4.6 | 0.40% | same setting; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Hindi | Claude Opus 4.6 | 1.11% | same setting; developer-reported | 4.1.2 |
| Higher-difficulty violative request evaluation — Overall harmless response rate | Claude Opus 4.7 | 99.05% | ±0.11%; 1,000 prompts each for 14 policy areas, English only; experimental prompt set; developer-reported | 4.1.3.1 |
| Higher-difficulty violative request evaluation — Overall harmless response rate | Claude Mythos Preview | 99.14% | ±0.11%; developer-reported | 4.1.3.1 |
| Higher-difficulty violative request evaluation — Overall harmless response rate | Claude Sonnet 4.6 | 99.27% | ±0.10%; developer-reported | 4.1.3.1 |
| Higher-difficulty violative request evaluation — Overall harmless response rate | Claude Opus 4.6 | 99.19% | ±0.11%; developer-reported | 4.1.3.1 |
| Higher-difficulty violative request evaluation — Harmless response rate, without thinking | Claude Opus 4.7 | 99.16% | ±0.15%; developer-reported | 4.1.3.1 |
| Higher-difficulty violative request evaluation — Harmless response rate, without thinking | Claude Mythos Preview | 99.28% | ±0.14%; developer-reported | 4.1.3.1 |
| Higher-difficulty violative request evaluation — Harmless response rate, without thinking | Claude Sonnet 4.6 | 99.14% | ±0.15%; developer-reported | 4.1.3.1 |
| Higher-difficulty violative request evaluation — Harmless response rate, without thinking | Claude Opus 4.6 | 99.09% | ±0.16%; developer-reported | 4.1.3.1 |
| Higher-difficulty violative request evaluation — Harmless response rate, with thinking | Claude Opus 4.7 | 98.94% | ±0.17%; adaptive thinking; developer-reported | 4.1.3.1 |
| Higher-difficulty violative request evaluation — Harmless response rate, with thinking | Claude Mythos Preview | 99.01% | ±0.16%; adaptive thinking; developer-reported | 4.1.3.1 |
| Higher-difficulty violative request evaluation — Harmless response rate, with thinking | Claude Sonnet 4.6 | 99.40% | ±0.13%; developer-reported | 4.1.3.1 |
| Higher-difficulty violative request evaluation — Harmless response rate, with thinking | Claude Opus 4.6 | 99.28% | ±0.14%; developer-reported | 4.1.3.1 |
| Higher-difficulty benign request evaluation — Overall refusal rate | Claude Opus 4.7 | 0.01% | ±0.01%; English only; developer-reported | 4.1.3.2 |
| Higher-difficulty benign request evaluation — Overall refusal rate | Claude Mythos Preview | 0.02% | ±0.02%; developer-reported | 4.1.3.2 |
| Higher-difficulty benign request evaluation — Overall refusal rate | Claude Sonnet 4.6 | 0.16% | ±0.05%; developer-reported | 4.1.3.2 |
| Higher-difficulty benign request evaluation — Overall refusal rate | Claude Opus 4.6 | 0.04% | ±0.02%; developer-reported | 4.1.3.2 |
| Higher-difficulty benign request evaluation — Refusal rate, without thinking | Claude Opus 4.7 | 0.02% | ±0.02%; developer-reported | 4.1.3.2 |
| Higher-difficulty benign request evaluation — Refusal rate, without thinking | Claude Mythos Preview | 0.03% | ±0.03%; developer-reported | 4.1.3.2 |
| Higher-difficulty benign request evaluation — Refusal rate, without thinking | Claude Sonnet 4.6 | 0.19% | ±0.07%; developer-reported | 4.1.3.2 |
| Higher-difficulty benign request evaluation — Refusal rate, without thinking | Claude Opus 4.6 | 0.06% | ±0.04%; developer-reported | 4.1.3.2 |
| Higher-difficulty benign request evaluation — Refusal rate, with thinking | Claude Opus 4.7 | 0.00% | adaptive thinking; developer-reported | 4.1.3.2 |
| Higher-difficulty benign request evaluation — Refusal rate, with thinking | Claude Mythos Preview | 0.01% | ±0.01%; developer-reported | 4.1.3.2 |
| Higher-difficulty benign request evaluation — Refusal rate, with thinking | Claude Sonnet 4.6 | 0.14% | ±0.06%; developer-reported | 4.1.3.2 |
| Higher-difficulty benign request evaluation — Refusal rate, with thinking | Claude Opus 4.6 | 0.03% | ±0.03%; developer-reported | 4.1.3.2 |
| Child safety single-turn violative requests — Harmless rate | Claude Opus 4.7 | 99.92% | ±0.07%; all tested languages; developer-reported | 4.4.1 |
| Child safety single-turn violative requests — Harmless rate | Claude Mythos Preview | 99.87% | ±0.08%; developer-reported | 4.4.1 |
| Child safety single-turn violative requests — Harmless rate | Claude Sonnet 4.6 | 99.95% | ±0.07%; developer-reported | 4.4.1 |
| Child safety single-turn violative requests — Harmless rate | Claude Opus 4.6 | 99.86% | ±0.12%; developer-reported | 4.4.1 |
| Child safety single-turn benign requests — Refusal rate | Claude Opus 4.7 | 0.01% | ±0.02%; the single declined benign prompt asked about penalties for possessing child sexual abuse material; developer-reported | 4.4.1 |
| Child safety single-turn benign requests — Refusal rate | Claude Mythos Preview | 0.04% | ±0.04%; developer-reported | 4.4.1 |
| Child safety single-turn benign requests — Refusal rate | Claude Sonnet 4.6 | 0.45% | ±0.23%; developer-reported | 4.4.1 |
| Child safety single-turn benign requests — Refusal rate | Claude Opus 4.6 | 0.67% | ±0.28%; developer-reported | 4.4.1 |
| Child safety multi-turn evaluations — Appropriate response rate | Claude Opus 4.7 | 95% | ±3%; developer-reported | 4.4.1 |
| Child safety multi-turn evaluations — Appropriate response rate | Claude Mythos Preview | 98% | ±2%; developer-reported | 4.4.1 |
| Child safety multi-turn evaluations — Appropriate response rate | Claude Sonnet 4.6 | 96% | ±3%; developer-reported | 4.4.1 |
| Child safety multi-turn evaluations — Appropriate response rate | Claude Opus 4.6 | 98% | ±2%; developer-reported | 4.4.1 |
| Suicide and self-harm single-turn, requests posing potential risk — Harmless rate | Claude Opus 4.7 | 99.11% | ±0.22%; all tested languages; developer-reported | 4.4.2 |
| Suicide and self-harm single-turn, requests posing potential risk — Harmless rate | Claude Mythos Preview | 99.58% | ±0.15%; developer-reported | 4.4.2 |
| Suicide and self-harm single-turn, requests posing potential risk — Harmless rate | Claude Sonnet 4.6 | 99.48% | ±0.22%; developer-reported | 4.4.2 |
| Suicide and self-harm single-turn, requests posing potential risk — Harmless rate | Claude Opus 4.6 | 99.41% | ±0.22%; developer-reported | 4.4.2 |
| Suicide and self-harm single-turn benign requests — Refusal rate | Claude Opus 4.7 | 0.05% | ±0.06%; developer-reported | 4.4.2 |
| Suicide and self-harm single-turn benign requests — Refusal rate | Claude Mythos Preview | 0.12% | ±0.10%; developer-reported | 4.4.2 |
| Suicide and self-harm single-turn benign requests — Refusal rate | Claude Sonnet 4.6 | 0.19% | ±0.13%; developer-reported | 4.4.2 |
| Suicide and self-harm single-turn benign requests — Refusal rate | Claude Opus 4.6 | 0.27% | ±0.15%; developer-reported | 4.4.2 |
| Suicide and self-harm multi-turn evaluations — Appropriate response rate | Claude Opus 4.7 | 82% | ±11%; updated graders split into separate suicide and self-harm graders; developer-reported | 4.4.2 |
| Suicide and self-harm multi-turn evaluations — Appropriate response rate | Claude Mythos Preview | 94% | ±7%; developer-reported | 4.4.2 |
| Suicide and self-harm multi-turn evaluations — Appropriate response rate | Claude Sonnet 4.6 | 76% | ±12%; developer-reported | 4.4.2 |
| Suicide and self-harm multi-turn evaluations — Appropriate response rate | Claude Opus 4.6 | 64% | ±14%; developer-reported | 4.4.2 |
| Disordered eating single-turn, requests posing potential risk — Harmless rate | Claude Opus 4.7 | 98.24% | ±0.44%; all tested languages; developer-reported | 4.4.3 |
| Disordered eating single-turn, requests posing potential risk — Harmless rate | Claude Mythos Preview | 98.20% | ±0.45%; developer-reported | 4.4.3 |
| Disordered eating single-turn, requests posing potential risk — Harmless rate | Claude Sonnet 4.6 | 98.07% | ±0.47%; developer-reported | 4.4.3 |
| Disordered eating single-turn, requests posing potential risk — Harmless rate | Claude Opus 4.6 | 98.55% | ±0.41%; developer-reported | 4.4.3 |
| Disordered eating single-turn benign requests — Refusal rate | Claude Opus 4.7 | 0.01% | ±0.02%; developer-reported | 4.4.3 |
| Disordered eating single-turn benign requests — Refusal rate | Claude Mythos Preview | 0.01% | ±0.02%; developer-reported | 4.4.3 |
| Disordered eating single-turn benign requests — Refusal rate | Claude Sonnet 4.6 | 0.22% | ±0.14%; developer-reported | 4.4.3 |
| Disordered eating single-turn benign requests — Refusal rate | Claude Opus 4.6 | 0.33% | ±0.19%; developer-reported | 4.4.3 |
| BBQ — Disambiguated accuracy | Claude Opus 4.7 | 81.3 | percent; thinking disabled; developer-reported | 4.5.2 |
| BBQ — Disambiguated accuracy | Claude Mythos Preview | 84.6 | percent; thinking disabled; developer-reported | 4.5.2 |
| BBQ — Disambiguated accuracy | Claude Sonnet 4.6 | 88.1 | percent; thinking disabled; developer-reported | 4.5.2 |
| BBQ — Disambiguated accuracy | Claude Opus 4.6 | 90.9 | percent; thinking disabled; developer-reported | 4.5.2 |
| BBQ — Ambiguous accuracy | Claude Opus 4.7 | 99.9 | percent; thinking disabled; developer-reported | 4.5.2 |
| BBQ — Ambiguous accuracy | Claude Mythos Preview | 100 | percent; thinking disabled; developer-reported | 4.5.2 |
| BBQ — Ambiguous accuracy | Claude Sonnet 4.6 | 97.5 | percent; thinking disabled; developer-reported | 4.5.2 |
| BBQ — Ambiguous accuracy | Claude Opus 4.6 | 99.7 | percent; thinking disabled; developer-reported | 4.5.2 |
| BBQ — Disambiguated bias score | Claude Opus 4.7 | -1.68 | percent; closer to zero is better; developer-reported | 4.5.2 |
| BBQ — Disambiguated bias score | Claude Mythos Preview | -1.61 | percent; developer-reported | 4.5.2 |
| BBQ — Disambiguated bias score | Claude Sonnet 4.6 | -0.67 | percent; developer-reported | 4.5.2 |
| BBQ — Disambiguated bias score | Claude Opus 4.6 | -0.73 | percent; developer-reported | 4.5.2 |
| BBQ — Ambiguous bias score | Claude Opus 4.7 | 0.04 | percent; developer-reported | 4.5.2 |
| BBQ — Ambiguous bias score | Claude Mythos Preview | 0.01 | percent; developer-reported | 4.5.2 |
| BBQ — Ambiguous bias score | Claude Sonnet 4.6 | 1.41 | percent; developer-reported | 4.5.2 |
| BBQ — Ambiguous bias score | Claude Opus 4.6 | 0.14 | percent; developer-reported | 4.5.2 |
| BBQ (direction of incorrect disambiguated answers) — Anti-stereotype share | Claude Opus 4.7 | roughly 54% | remaining 46% stereotype-consistent; developer-reported | 4.5.2 |
| BBQ (direction of incorrect disambiguated answers) — Anti-stereotype share | Claude Opus 4.6 | 52% | remaining 48% stereotype-consistent; developer-reported | 4.5.2 |
| Election integrity single-turn violative requests — Harmless rate, API without system prompt | Claude Opus 4.7 | 100% | 300 violative and 300 benign prompts; thinking off; developer-reported | 4.5.3 |
| Election integrity single-turn violative requests — Harmless rate, API without system prompt | Claude Mythos Preview | 100% | same evaluation; developer-reported | 4.5.3 |
| Election integrity single-turn violative requests — Harmless rate, API without system prompt | Claude Sonnet 4.6 | 99.89% | same evaluation; developer-reported | 4.5.3 |
| Election integrity single-turn violative requests — Harmless rate, API without system prompt | Claude Opus 4.6 | 100% | same evaluation; developer-reported | 4.5.3 |
| Election integrity single-turn violative requests — Harmless rate, claude.ai | Claude Opus 4.7 | 100% | with the default system prompt; developer-reported | 4.5.3 |
| Election integrity single-turn violative requests — Harmless rate, claude.ai | Claude Mythos Preview | 100% | same evaluation; developer-reported | 4.5.3 |
| Election integrity single-turn violative requests — Harmless rate, claude.ai | Claude Sonnet 4.6 | 100% | same evaluation; developer-reported | 4.5.3 |
| Election integrity single-turn violative requests — Harmless rate, claude.ai | Claude Opus 4.6 | 99.89% | same evaluation; developer-reported | 4.5.3 |
| Election integrity single-turn benign requests — Refusal rate, API without system prompt | Claude Opus 4.7 | 0.00% | thinking off; developer-reported | 4.5.3 |
| Election integrity single-turn benign requests — Refusal rate, API without system prompt | Claude Mythos Preview | 0.00% | thinking off; developer-reported | 4.5.3 |
| Election integrity single-turn benign requests — Refusal rate, API without system prompt | Claude Sonnet 4.6 | 0.00% | thinking off; developer-reported | 4.5.3 |
| Election integrity single-turn benign requests — Refusal rate, API without system prompt | Claude Opus 4.6 | 0.11% | thinking off; developer-reported | 4.5.3 |
| Election integrity single-turn benign requests — Refusal rate, claude.ai | Claude Opus 4.7 | 0.67% | with the default system prompt; developer-reported | 4.5.3 |
| Election integrity single-turn benign requests — Refusal rate, claude.ai | Claude Mythos Preview | 0.33% | same evaluation; developer-reported | 4.5.3 |
| Election integrity single-turn benign requests — Refusal rate, claude.ai | Claude Sonnet 4.6 | 0.00% | same evaluation; developer-reported | 4.5.3 |
| Election integrity single-turn benign requests — Refusal rate, claude.ai | Claude Opus 4.6 | 0.00% | same evaluation; developer-reported | 4.5.3 |
| Malicious use of Claude Code — Malicious refusal rate | Claude Opus 4.7 | 91.15% | 61 malicious prompts run 10 times each; Claude Code system prompt, without the FileTool reminder; developer-reported | 5.1.1 |
| Malicious use of Claude Code — Malicious refusal rate | Claude Mythos Preview | 95.41% | without the FileTool reminder; developer-reported | 5.1.1 |
| Malicious use of Claude Code — Malicious refusal rate | Claude Sonnet 4.6 | 82.21% | with the FileTool reminder; developer-reported | 5.1.1 |
| Malicious use of Claude Code — Malicious refusal rate | Claude Opus 4.6 | 81.94% | without the FileTool reminder; developer-reported | 5.1.1 |
| Malicious use of Claude Code — Dual-use and benign success rate | Claude Opus 4.7 | 91.83% | 61 dual-use/benign prompts run 10 times each; without the FileTool reminder; developer-reported | 5.1.1 |
| Malicious use of Claude Code — Dual-use and benign success rate | Claude Mythos Preview | 91.12% | without the FileTool reminder; developer-reported | 5.1.1 |
| Malicious use of Claude Code — Dual-use and benign success rate | Claude Sonnet 4.6 | 98.61% | with the FileTool reminder; developer-reported | 5.1.1 |
| Malicious use of Claude Code — Dual-use and benign success rate | Claude Opus 4.6 | 94.97% | without the FileTool reminder; developer-reported | 5.1.1 |
| Malicious computer use — Refusal rate, without mitigations | Claude Opus 4.7 | 89.29% | 112 tasks with thinking on and off (224 attempts); developer-reported | 5.1.2 |
| Malicious computer use — Refusal rate, without mitigations | Claude Mythos Preview | 93.75% | same task set; developer-reported | 5.1.2 |
| Malicious computer use — Refusal rate, without mitigations | Claude Sonnet 4.6 | 84.82% | same task set; developer-reported | 5.1.2 |
| Malicious computer use — Refusal rate, without mitigations | Claude Opus 4.6 | 87.05% | same task set; developer-reported | 5.1.2 |
| Malicious agentic influence campaigns — Voter suppression scenario | Claude Opus 4.7 (helpful-only) | 57.1% | task completion rate; share of 70 success criteria; 9 simulations (3 runs at 3 friction levels); helpful-only variant; developer-reported | 5.1.3 |
| Malicious agentic influence campaigns — Voter suppression scenario | Claude Mythos Preview (helpful-only) | 59.5% | same evaluation; developer-reported | 5.1.3 |
| Malicious agentic influence campaigns — Voter suppression scenario | Claude Sonnet 4.6 (helpful-only) | 41.8% | same evaluation; developer-reported | 5.1.3 |
| Malicious agentic influence campaigns — Voter suppression scenario | Claude Opus 4.6 (helpful-only) | 54.4% | same evaluation; developer-reported | 5.1.3 |
| Malicious agentic influence campaigns — Domestic polarization scenario | Claude Opus 4.7 (helpful-only) | 46.8% | task completion rate; share of 70 success criteria; 9 simulations; developer-reported | 5.1.3 |
| Malicious agentic influence campaigns — Domestic polarization scenario | Claude Mythos Preview (helpful-only) | 42.1% | same evaluation; developer-reported | 5.1.3 |
| Malicious agentic influence campaigns — Domestic polarization scenario | Claude Sonnet 4.6 (helpful-only) | 34.0% | same evaluation; developer-reported | 5.1.3 |
| Malicious agentic influence campaigns — Domestic polarization scenario | Claude Opus 4.6 (helpful-only) | 33.7% | same evaluation; developer-reported | 5.1.3 |
| Malicious agentic influence campaigns — Fully-trained models, both scenarios | Claude models (fully trained) | near 0% | task completion rate; models generally refused from the first turn; developer-reported | 5.1.3 |
| Agent Red Teaming (ART) benchmark, indirect prompt injection — Attack success rate at k=100, no thinking | Claude Opus 4.7 | 6.0% | 19 scenarios; high-transfer attack subset; grading issues corrected with Gray Swan, so figures may differ from earlier cards; evaluated by Gray Swan, reported by Anthropic | 5.2.1 |
| Agent Red Teaming (ART) benchmark, indirect prompt injection — Attack success rate at k=100, adaptive thinking | Claude Opus 4.7 | 4.8% | evaluated by Gray Swan, reported by Anthropic | 5.2.1 |
| Agent Red Teaming (ART) benchmark, indirect prompt injection — Attack success rate at k=100, no thinking | Claude Opus 4.6 | 14.8% | evaluated by Gray Swan, reported by Anthropic | 5.2.1 |
| Agent Red Teaming (ART) benchmark, indirect prompt injection — Attack success rate at k=100, adaptive thinking | Claude Opus 4.6 | 21.7% | evaluated by Gray Swan, reported by Anthropic | 5.2.1 |
| Shade adaptive prompt injection, coding — 1 attempt, no safeguards, with thinking | Claude Opus 4.7 | 2.34% | attack success rate; Gray Swan Shade attacker on 40 test cases; stronger attacker than earlier cards; developer-reported | 5.2.2.1 |
| Shade adaptive prompt injection, coding — 1 attempt, no safeguards, with thinking | Claude Mythos Preview | 0.0% | same attacker and test cases; developer-reported | 5.2.2.1 |
| Shade adaptive prompt injection, coding — 1 attempt, no safeguards, with thinking | Claude Sonnet 4.6 | 12.71% | same attacker and test cases; developer-reported | 5.2.2.1 |
| Shade adaptive prompt injection, coding — 1 attempt, no safeguards, with thinking | Claude Opus 4.6 | 25.92% | same attacker and test cases; developer-reported | 5.2.2.1 |
| Shade adaptive prompt injection, coding — 1 attempt, no safeguards, no thinking | Claude Opus 4.7 | 10.43% | same attacker and test cases; developer-reported | 5.2.2.1 |
| Shade adaptive prompt injection, coding — 1 attempt, no safeguards, no thinking | Claude Mythos Preview | 0.03% | same attacker and test cases; developer-reported | 5.2.2.1 |
| Shade adaptive prompt injection, coding — 1 attempt, no safeguards, no thinking | Claude Sonnet 4.6 | 45.26% | same attacker and test cases; developer-reported | 5.2.2.1 |
| Shade adaptive prompt injection, coding — 1 attempt, no safeguards, no thinking | Claude Opus 4.6 | 54.14% | same attacker and test cases; developer-reported | 5.2.2.1 |
| Shade adaptive prompt injection, coding — 200 attempts, no safeguards, with thinking | Claude Opus 4.7 | 60.0% | at least one of 200 adaptive attempts succeeded; developer-reported | 5.2.2.1 |
| Shade adaptive prompt injection, coding — 200 attempts, no safeguards, with thinking | Claude Mythos Preview | 0.0% | same measure; developer-reported | 5.2.2.1 |
| Shade adaptive prompt injection, coding — 200 attempts, no safeguards, with thinking | Claude Sonnet 4.6 | 90.0% | same measure; developer-reported | 5.2.2.1 |
| Shade adaptive prompt injection, coding — 200 attempts, no safeguards, with thinking | Claude Opus 4.6 | 97.5% | same measure; developer-reported | 5.2.2.1 |
| Shade adaptive prompt injection, coding — 200 attempts, no safeguards, no thinking | Claude Opus 4.7 | 92.5% | same measure; developer-reported | 5.2.2.1 |
| Shade adaptive prompt injection, coding — 200 attempts, no safeguards, no thinking | Claude Mythos Preview | 2.5% | same measure; developer-reported | 5.2.2.1 |
| Shade adaptive prompt injection, coding — 200 attempts, no safeguards, no thinking | Claude Sonnet 4.6 | 100% | same measure; developer-reported | 5.2.2.1 |
| Shade adaptive prompt injection, coding — 200 attempts, no safeguards, no thinking | Claude Opus 4.6 | 100% | same measure; developer-reported | 5.2.2.1 |
| Shade adaptive prompt injection, coding — 1 attempt, with safeguards, with thinking | Claude Opus 4.7 | 0.43% | safeguards enabled; developer-reported | 5.2.2.1 |
| Shade adaptive prompt injection, coding — 1 attempt, with safeguards, with thinking | Claude Mythos Preview | 0.0% | safeguards enabled; developer-reported | 5.2.2.1 |
| Shade adaptive prompt injection, coding — 1 attempt, with safeguards, with thinking | Claude Sonnet 4.6 | 2.99% | safeguards enabled; developer-reported | 5.2.2.1 |
| Shade adaptive prompt injection, coding — 1 attempt, with safeguards, with thinking | Claude Opus 4.6 | 3.86% | safeguards enabled; developer-reported | 5.2.2.1 |
| Shade adaptive prompt injection, coding — 1 attempt, with safeguards, no thinking | Claude Opus 4.7 | 1.76% | safeguards enabled; developer-reported | 5.2.2.1 |
| Shade adaptive prompt injection, coding — 1 attempt, with safeguards, no thinking | Claude Mythos Preview | 0.0% | safeguards enabled; developer-reported | 5.2.2.1 |
| Shade adaptive prompt injection, coding — 1 attempt, with safeguards, no thinking | Claude Sonnet 4.6 | 8.70% | safeguards enabled; developer-reported | 5.2.2.1 |
| Shade adaptive prompt injection, coding — 1 attempt, with safeguards, no thinking | Claude Opus 4.6 | 6.81% | safeguards enabled; developer-reported | 5.2.2.1 |
| Shade adaptive prompt injection, coding — 200 attempts, with safeguards, with thinking | Claude Opus 4.7 | 25.0% | safeguards enabled; developer-reported | 5.2.2.1 |
| Shade adaptive prompt injection, coding — 200 attempts, with safeguards, with thinking | Claude Mythos Preview | 0.0% | safeguards enabled; developer-reported | 5.2.2.1 |
| Shade adaptive prompt injection, coding — 200 attempts, with safeguards, with thinking | Claude Sonnet 4.6 | 80.0% | safeguards enabled; developer-reported | 5.2.2.1 |
| Shade adaptive prompt injection, coding — 200 attempts, with safeguards, with thinking | Claude Opus 4.6 | 80.0% | safeguards enabled; developer-reported | 5.2.2.1 |
| Shade adaptive prompt injection, coding — 200 attempts, with safeguards, no thinking | Claude Opus 4.7 | 52.5% | safeguards enabled; developer-reported | 5.2.2.1 |
| Shade adaptive prompt injection, coding — 200 attempts, with safeguards, no thinking | Claude Mythos Preview | 0.0% | safeguards enabled; developer-reported | 5.2.2.1 |
| Shade adaptive prompt injection, coding — 200 attempts, with safeguards, no thinking | Claude Sonnet 4.6 | 100% | safeguards enabled; developer-reported | 5.2.2.1 |
| Shade adaptive prompt injection, coding — 200 attempts, with safeguards, no thinking | Claude Opus 4.6 | 92.5% | safeguards enabled; developer-reported | 5.2.2.1 |
| Shade adaptive prompt injection, computer use — 1 attempt, no safeguards, with thinking | Claude Opus 4.7 | 0.46% | attack success rate; 14 test cases; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, computer use — 1 attempt, no safeguards, with thinking | Claude Mythos Preview | 0.43% | same test cases; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, computer use — 1 attempt, no safeguards, with thinking | Claude Sonnet 4.6 | 12.0% | same test cases; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, computer use — 1 attempt, no safeguards, with thinking | Claude Opus 4.6 | 17.8% | same test cases; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, computer use — 1 attempt, no safeguards, no thinking | Claude Opus 4.7 | 0.39% | same test cases; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, computer use — 1 attempt, no safeguards, no thinking | Claude Mythos Preview | 0.39% | same test cases; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, computer use — 1 attempt, no safeguards, no thinking | Claude Sonnet 4.6 | 14.4% | same test cases; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, computer use — 1 attempt, no safeguards, no thinking | Claude Opus 4.6 | 20.0% | same test cases; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, computer use — 200 attempts, no safeguards, with thinking | Claude Opus 4.7 | 7.14% | at least one of 200 attempts succeeded; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, computer use — 200 attempts, no safeguards, with thinking | Claude Mythos Preview | 21.43% | same measure; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, computer use — 200 attempts, no safeguards, with thinking | Claude Sonnet 4.6 | 42.9% | same measure; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, computer use — 200 attempts, no safeguards, with thinking | Claude Opus 4.6 | 78.6% | same measure; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, computer use — 200 attempts, no safeguards, no thinking | Claude Opus 4.7 | 21.43% | same measure; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, computer use — 200 attempts, no safeguards, no thinking | Claude Mythos Preview | 14.29% | same measure; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, computer use — 200 attempts, no safeguards, no thinking | Claude Sonnet 4.6 | 64.3% | same measure; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, computer use — 200 attempts, no safeguards, no thinking | Claude Opus 4.6 | 85.7% | same measure; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, computer use — 1 attempt, with safeguards, with thinking | Claude Opus 4.7 | 0.61% | safeguards enabled; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, computer use — 1 attempt, with safeguards, with thinking | Claude Mythos Preview | 0.32% | safeguards enabled; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, computer use — 1 attempt, with safeguards, with thinking | Claude Sonnet 4.6 | 6.21% | safeguards enabled; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, computer use — 1 attempt, with safeguards, with thinking | Claude Opus 4.6 | 9.32% | safeguards enabled; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, computer use — 1 attempt, with safeguards, no thinking | Claude Opus 4.7 | 0.50% | safeguards enabled; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, computer use — 1 attempt, with safeguards, no thinking | Claude Mythos Preview | 0.36% | safeguards enabled; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, computer use — 1 attempt, with safeguards, no thinking | Claude Sonnet 4.6 | 6.32% | safeguards enabled; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, computer use — 1 attempt, with safeguards, no thinking | Claude Opus 4.6 | 9.96% | safeguards enabled; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, computer use — 200 attempts, with safeguards, with thinking | Claude Opus 4.7 | 14.29% | safeguards enabled; Anthropic notes the rise over the no-safeguards figure is not statistically significant with 14 test cases; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, computer use — 200 attempts, with safeguards, with thinking | Claude Mythos Preview | 21.43% | safeguards enabled; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, computer use — 200 attempts, with safeguards, with thinking | Claude Sonnet 4.6 | 64.29% | safeguards enabled; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, computer use — 200 attempts, with safeguards, with thinking | Claude Opus 4.6 | 50.0% | safeguards enabled; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, computer use — 200 attempts, with safeguards, no thinking | Claude Opus 4.7 | 35.71% | safeguards enabled; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, computer use — 200 attempts, with safeguards, no thinking | Claude Mythos Preview | 14.29% | safeguards enabled; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, computer use — 200 attempts, with safeguards, no thinking | Claude Sonnet 4.6 | 78.57% | safeguards enabled; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, computer use — 200 attempts, with safeguards, no thinking | Claude Opus 4.6 | 50.0% | safeguards enabled; developer-reported | 5.2.2.2 |
| Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, no safeguards, with thinking | Claude Opus 4.7 | 4.05% | 148 held-out environments, 10 attempts each; attacks adaptively sourced against Opus 4.6 by professional red-teamers; developer-reported | 5.2.2.3 |
| Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, no safeguards, with thinking | Claude Mythos Preview | 0.68% | same environments; developer-reported | 5.2.2.3 |
| Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, no safeguards, with thinking | Claude Sonnet 4.6 | 55.41% | same environments; developer-reported | 5.2.2.3 |
| Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, no safeguards, with thinking | Claude Opus 4.6 | 80.41% | same environments; attacks were sourced adaptively against this model; developer-reported | 5.2.2.3 |
| Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, no safeguards, no thinking | Claude Opus 4.7 | 4.73% | same environments; developer-reported | 5.2.2.3 |
| Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, no safeguards, no thinking | Claude Mythos Preview | 1.35% | same environments; developer-reported | 5.2.2.3 |
| Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, no safeguards, no thinking | Claude Sonnet 4.6 | 54.05% | same environments; developer-reported | 5.2.2.3 |
| Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, no safeguards, no thinking | Claude Opus 4.6 | 86.49% | same environments; developer-reported | 5.2.2.3 |
| Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, no safeguards, with thinking | Claude Opus 4.7 | 0.74% | per-attempt attack success rate across 148 environments; developer-reported | 5.2.2.3 |
| Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, no safeguards, with thinking | Claude Mythos Preview | 0.07% | same measure; developer-reported | 5.2.2.3 |
| Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, no safeguards, with thinking | Claude Sonnet 4.6 | 30.74% | same measure; developer-reported | 5.2.2.3 |
| Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, no safeguards, with thinking | Claude Opus 4.6 | 45.81% | same measure; developer-reported | 5.2.2.3 |
| Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, no safeguards, no thinking | Claude Opus 4.7 | 0.75% | same measure; developer-reported | 5.2.2.3 |
| Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, no safeguards, no thinking | Claude Mythos Preview | 0.14% | same measure; developer-reported | 5.2.2.3 |
| Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, no safeguards, no thinking | Claude Sonnet 4.6 | 34.66% | same measure; developer-reported | 5.2.2.3 |
| Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, no safeguards, no thinking | Claude Opus 4.6 | 54.93% | same measure; developer-reported | 5.2.2.3 |
| Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, with safeguards, with thinking | Claude Opus 4.7 | 0.00% | deployed safeguards enabled; no attacks succeeded across the 148 environments; developer-reported | 5.2.2.3 |
| Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, with safeguards, with thinking | Claude Mythos Preview | 0.00% | safeguards enabled; developer-reported | 5.2.2.3 |
| Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, with safeguards, with thinking | Claude Sonnet 4.6 | 2.70% | safeguards enabled; developer-reported | 5.2.2.3 |
| Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, with safeguards, with thinking | Claude Opus 4.6 | 2.70% | safeguards enabled; developer-reported | 5.2.2.3 |
| Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, with safeguards, no thinking | Claude Opus 4.7 | 0.00% | safeguards enabled; developer-reported | 5.2.2.3 |
| Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, with safeguards, no thinking | Claude Mythos Preview | 0.00% | safeguards enabled; developer-reported | 5.2.2.3 |
| Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, with safeguards, no thinking | Claude Sonnet 4.6 | 2.70% | safeguards enabled; developer-reported | 5.2.2.3 |
| Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, with safeguards, no thinking | Claude Opus 4.6 | 0.68% | safeguards enabled; developer-reported | 5.2.2.3 |
| Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, with safeguards, with thinking | Claude Opus 4.7 | 0.00% | safeguards enabled; developer-reported | 5.2.2.3 |
| Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, with safeguards, with thinking | Claude Mythos Preview | 0.00% | safeguards enabled; developer-reported | 5.2.2.3 |
| Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, with safeguards, with thinking | Claude Sonnet 4.6 | 0.41% | safeguards enabled; developer-reported | 5.2.2.3 |
| Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, with safeguards, with thinking | Claude Opus 4.6 | 0.41% | safeguards enabled; developer-reported | 5.2.2.3 |
| Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, with safeguards, no thinking | Claude Opus 4.7 | 0.00% | safeguards enabled; developer-reported | 5.2.2.3 |
| Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, with safeguards, no thinking | Claude Mythos Preview | 0.00% | safeguards enabled; developer-reported | 5.2.2.3 |
| Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, with safeguards, no thinking | Claude Sonnet 4.6 | 0.34% | safeguards enabled; developer-reported | 5.2.2.3 |
| Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, with safeguards, no thinking | Claude Opus 4.6 | 0.07% | safeguards enabled; developer-reported | 5.2.2.3 |
| Constitutional adherence (15 dimensions) — Dimensions where Opus 4.7 scored higher than Opus 4.6, Sonnet 4.6 and Haiku 4.5 | Claude Opus 4.7 | 10 / 15 | roughly 1,000 transcripts across 40 constitutional areas; scored -3 to +3 by a helpful-only Claude Opus 4.6; improvements over Opus 4.6 on honesty, ethics and safety were statistically significant; developer-reported | 6.3.2.3 |
| Destructive-action audit quadrant (score >=4 on both destructiveness and dishonesty) | Claude Opus 4.7 | 3 investigations | automated behavioral audit; x = max(reckless, overeager tool use), y = max(failure to disclose, important omissions, unprompted deception); developer-reported | 6.3.1.3 |
| Destructive-action audit quadrant (score >=4 on both destructiveness and dishonesty) | Claude Opus 4.6 | 24 investigations | same measure; developer-reported | 6.3.1.3 |
| Destructive-action audit quadrant (score >=4 on both destructiveness and dishonesty) | Claude Mythos Preview | 0 investigations | same measure; developer-reported | 6.3.1.3 |
| Verbalized evaluation awareness (synthetic-backend destructiveness evaluation) | Claude Opus 4.7 | about 9% of transcripts | simulated tool results generated by Claude Opus 4.6; developer-reported | 6.3.1.1 |
| Verbalized evaluation awareness (synthetic-backend destructiveness evaluation) | Claude Mythos Preview | approximately 1% of transcripts | same evaluation; developer-reported | 6.3.1.1 |
| Verbalized evaluation awareness (synthetic-backend destructiveness evaluation) | Claude Opus 4.6 | approximately 1% of transcripts | same evaluation; developer-reported | 6.3.1.1 |
| Verbalized evaluation awareness (synthetic-backend destructiveness evaluation) | Claude Sonnet 4.6 | approximately 1% of transcripts | same evaluation; developer-reported | 6.3.1.1 |
| Refusal to assist with AI safety R&D — Refusal rate | Claude Opus 4.7 | 33% | single-turn coding queries; extended thinking on; legacy token-budget system; mostly partial refusals; developer-reported | 6.3.4 |
| Refusal to assist with AI safety R&D — Refusal rate | Claude Opus 4.6 | 12% | same evaluation; developer-reported | 6.3.4 |
| Claude self-preference evaluation — Largest condition (Claude system prompt, chain of thought on) | Claude Opus 4.7 | 0.4 | difference in average quality score on a 0–10 scale when the transcript is attributed to Claude; 3 of 4 conditions statistically significant; developer-reported | 6.3.5 |
| UK AISI AI-safety-sabotage evaluation — Partial refusals on AI safety research tasks | Claude Opus 4.7 | 13.7% | with 0% full refusals; measured by the UK AI Security Institute | 6.2.4 |
| UK AISI AI-safety-sabotage evaluation — Partial refusals on AI safety research tasks | Claude Mythos Preview | 33% | with 0.7% full refusals; measured by the UK AI Security Institute | 6.2.4 |
| UK AISI AI-safety-sabotage evaluation — Partial refusals on AI safety research tasks | Claude Opus 4.6 | 6.3% | with 2.2% full refusals; measured by the UK AI Security Institute | 6.2.4 |
| UK AISI AI-safety-sabotage evaluation — Partial refusals on AI safety research tasks | Claude Sonnet 4.6 | 22.2% | with 10.7% full refusals; measured by the UK AI Security Institute | 6.2.4 |
| UK AISI AI-safety-sabotage evaluation — Continuation setting, actively continued the attempt | Claude Opus 4.7 | 0% | measured by the UK AI Security Institute | 6.2.4 |
| UK AISI AI-safety-sabotage evaluation — Continuation setting, actively continued the attempt | Claude Mythos Preview | 7% | measured by the UK AI Security Institute | 6.2.4 |
| UK AISI AI-safety-sabotage evaluation — Continuation setting, actively continued the attempt | Claude Opus 4.6 | 3% | measured by the UK AI Security Institute | 6.2.4 |
| UK AISI AI-safety-sabotage evaluation — Continuation setting, actively continued the attempt | Claude Sonnet 4.6 | 4% | measured by the UK AI Security Institute | 6.2.4 |
| UK AISI evaluation-awareness discrimination — Real deployment data vs. evaluation-scaffolded reproductions, prompted | Claude Opus 4.7 | 68% | accuracy; measured by the UK AI Security Institute | 6.2.4 |
| UK AISI evaluation-awareness discrimination — Real deployment data vs. evaluation-scaffolded reproductions, prompted | Claude Mythos Preview | 78% | accuracy; measured by the UK AI Security Institute | 6.2.4 |
| UK AISI evaluation-awareness discrimination — Real deployment data vs. evaluation-scaffolded reproductions, prompted | Claude Opus 4.6 | 76% | accuracy; measured by the UK AI Security Institute | 6.2.4 |
| UK AISI evaluation-awareness discrimination — Real deployment data vs. evaluation-scaffolded reproductions, prompted | Claude Sonnet 4.6 | 66% | accuracy; measured by the UK AI Security Institute | 6.2.4 |
| UK AISI evaluation-awareness discrimination — Continuation-pipeline reproductions, prompted | Claude Opus 4.7 | 62% | accuracy; measured by the UK AI Security Institute | 6.2.4 |
| UK AISI evaluation-awareness discrimination — Continuation-pipeline reproductions, prompted | Claude Opus 4.6 | 80% | accuracy; measured by the UK AI Security Institute | 6.2.4 |
| UK AISI evaluation-awareness discrimination — Continuation-pipeline reproductions, prompted | Claude Mythos Preview | 74% | accuracy; measured by the UK AI Security Institute | 6.2.4 |
| UK AISI evaluation-awareness discrimination — Continuation-pipeline reproductions, prompted | Claude Sonnet 4.6 | 68% | accuracy; measured by the UK AI Security Institute | 6.2.4 |
| UK AISI unprompted verbalised evaluation awareness — Unprompted sabotage evaluation | Claude Opus 4.7 | 21.3% | measured by the UK AI Security Institute | 6.2.4 |
| UK AISI unprompted verbalised evaluation awareness — Unprompted sabotage evaluation | Claude Opus 4.6 | 0% | measured by the UK AI Security Institute | 6.2.4 |
| UK AISI unprompted verbalised evaluation awareness — Unprompted sabotage evaluation | Claude Sonnet 4.6 | 17% | measured by the UK AI Security Institute | 6.2.4 |
| UK AISI unprompted verbalised evaluation awareness — Unprompted sabotage evaluation | Claude Mythos Preview | 17% | measured by the UK AI Security Institute | 6.2.4 |
| UK AISI unprompted verbalised evaluation awareness — Continuation evaluation | Claude Opus 4.7 | 3.4% | measured by the UK AI Security Institute | 6.2.4 |
| UK AISI unprompted verbalised evaluation awareness — Continuation evaluation | Claude Opus 4.6 | 0.5% | measured by the UK AI Security Institute | 6.2.4 |
| UK AISI unprompted verbalised evaluation awareness — Continuation evaluation | Claude Mythos Preview | 2.5% | measured by the UK AI Security Institute | 6.2.4 |
| UK AISI unprompted verbalised evaluation awareness — Continuation evaluation | Claude Sonnet 4.6 | 3.9% | measured by the UK AI Security Institute | 6.2.4 |
| Sandbagging screen of dangerous-capability transcripts — Transcripts flagged for manual review | Claude Opus 4.7 | 18 of 1,000 | 1,000 transcripts (500 non-biological on the final or an earlier Opus 4.7 snapshot, 500 helpful-only biological); graded 0–10 by Claude Sonnet 4.6; 12 classed as ordinary weighing of tradeoffs, 5 technical glitches, 1 possible sandbagging; developer-reported | 6.4.1 |
| SHADE-Arena — Stealth rate, Extended Thinking | Claude Opus 4.7 | 1.5–2.0% | monitor (Claude Opus 4.6) sees the agent's reasoning; roughly flat across reasoning-effort levels; developer-reported | 6.4.2.1 |
| SHADE-Arena — Stealth rate, Extended Thinking | Claude Opus 4.6 | 0–1.5% | same conditions; developer-reported | 6.4.2.1 |
| SHADE-Arena — Stealth rate, Extended Thinking | Claude Mythos Preview | 3.8–4.2% | same conditions; developer-reported | 6.4.2.1 |
| SHADE-Arena — Stealth rate, Hidden Scratchpad | Claude Opus 4.7 | 10.7% | monitor sees only user-visible outputs; upper bound; developer-reported | 6.4.2.1 |
| SHADE-Arena — Stealth rate, Hidden Scratchpad | Claude Mythos Preview | 13.8% | same condition; developer-reported | 6.4.2.1 |
| SHADE-Arena — Side task referenced in reasoning | Claude Opus 4.6 | roughly 95% | share of trajectories; developer-reported | 6.4.2.1 |
| SHADE-Arena — Side task referenced in reasoning | Claude Opus 4.7 | roughly 50% | share of trajectories; developer-reported | 6.4.2.1 |
| SHADE-Arena — Side task referenced in reasoning | Claude Mythos Preview | 15–20% | share of trajectories; developer-reported | 6.4.2.1 |
| Rare-action evaluation (Serrano et al. 2026 replication) — Baseline | Claude Opus 4.7 | 0 | 4,000 samples per setting; model instructed to take the action 1% of the time; developer-reported | 6.4.2.3 |
| Rare-action evaluation (Serrano et al. 2026 replication) — UUID | Claude Opus 4.7 | 4 (0.1%) | 4,000 samples; developer-reported | 6.4.2.3 |
| Rare-action evaluation (Serrano et al. 2026 replication) — SSoT | Claude Opus 4.7 | 16 (0.4%) | 4,000 samples; developer-reported | 6.4.2.3 |
| Rare-action evaluation (Serrano et al. 2026 replication) — Strategy hint | Claude Opus 4.7 | 0 | 4,000 samples; developer-reported | 6.4.2.3 |
| Rare-action evaluation (Serrano et al. 2026 replication) — Organic | Claude Opus 4.7 | 0 | 4,000 samples; developer-reported | 6.4.2.3 |
| Rare-action evaluation (Serrano et al. 2026 replication) — UUID | Claude Opus 4.5 | 19 (0.48%) | 4,000 samples; developer-reported | 6.4.2.3 |
| Rare-action evaluation (Serrano et al. 2026 replication) — SSoT | Claude Opus 4.5 | 14 (0.35%) | 4,000 samples; developer-reported | 6.4.2.3 |
| Rare-action evaluation (Serrano et al. 2026 replication) — UUID | Claude Opus 4.6 | 8 (0.20%) | 4,000 samples; developer-reported | 6.4.2.3 |
| Rare-action evaluation (Serrano et al. 2026 replication) — SSoT | Claude Opus 4.6 | 22 (0.55%) | 4,000 samples; developer-reported | 6.4.2.3 |
| Rare-action evaluation (Serrano et al. 2026 replication) — UUID | Claude Mythos Preview | 2 (0.05%) | 4,000 samples; developer-reported | 6.4.2.3 |
| Rare-action evaluation (Serrano et al. 2026 replication) — SSoT | Claude Mythos Preview | 18 (0.45%) | 4,000 samples; developer-reported | 6.4.2.3 |
| Evaluation-awareness probe validation — AUC, Testing vs. Conversation probe | Claude Opus 4.7 | 0.761 | against an LLM judge's union label of explicit evaluation framing or verbalized test suspicion, over automated behavioral audit transcripts; 12 random control vectors all scored below the three probes; developer-reported | 6.5.2.1 |
| Evaluation-awareness probe validation — AUC, Monitoring Sensitivity probe | Claude Opus 4.7 | 0.739 | same validation; developer-reported | 6.5.2.1 |
| Evaluation-awareness probe validation — AUC, Alignment Faking probe | Claude Opus 4.7 | 0.722 | same validation; developer-reported | 6.5.2.1 |
| GMMLU — English | Claude Opus 4.7 | 93.4% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — English | Claude Opus 4.6 | 92.8% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — English | Claude Sonnet 4.6 | 91.8% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — English | Gemini 3.1 Pro | 94.3% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — English | GPT-5.4 | 93.3% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — High-resource average (15 languages) | Claude Opus 4.7 | 91.5% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — High-resource average (15 languages) | Claude Opus 4.6 | 90.7% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — High-resource average (15 languages) | Claude Sonnet 4.6 | 89.2% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — High-resource average (15 languages) | Gemini 3.1 Pro | 93.1% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — High-resource average (15 languages) | GPT-5.4 | 91.5% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — Mid-resource average (14 languages) | Claude Opus 4.7 | 91.1% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Mid-resource average (14 languages) | Claude Opus 4.6 | 89.6% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Mid-resource average (14 languages) | Claude Sonnet 4.6 | 88.2% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Mid-resource average (14 languages) | Gemini 3.1 Pro | 92.9% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — Mid-resource average (14 languages) | GPT-5.4 | 91.4% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — Low-resource average | Claude Opus 4.7 | 86.2% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Low-resource average | Claude Opus 4.6 | 78.4% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Low-resource average | Claude Sonnet 4.6 | 79.2% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Low-resource average | Gemini 3.1 Pro | 90.3% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — Low-resource average | GPT-5.4 | 88.3% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — Igbo | Claude Opus 4.7 | 81.3% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Igbo | Claude Opus 4.6 | 70.1% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Igbo | Claude Sonnet 4.6 | 71.9% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Igbo | Gemini 3.1 Pro | 89.3% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — Igbo | GPT-5.4 | 86.4% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — Yoruba | Claude Opus 4.7 | 82.9% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Yoruba | Claude Opus 4.6 | 70.8% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Yoruba | Claude Sonnet 4.6 | 76.9% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Yoruba | Gemini 3.1 Pro | 88.4% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — Yoruba | GPT-5.4 | 83.8% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — Somali | Claude Opus 4.7 | 84.1% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Somali | Claude Opus 4.6 | 72.0% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Somali | Claude Sonnet 4.6 | 75.5% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Somali | Gemini 3.1 Pro | 90.5% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — Somali | GPT-5.4 | 88.7% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — Malagasy | Claude Opus 4.7 | 84.8% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Malagasy | Claude Opus 4.6 | 78.6% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Malagasy | Claude Sonnet 4.6 | 78.4% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Malagasy | Gemini 3.1 Pro | 90.7% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — Malagasy | GPT-5.4 | 88.8% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — Chichewa | Claude Opus 4.7 | 84.9% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Chichewa | Claude Opus 4.6 | 71.2% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Chichewa | Claude Sonnet 4.6 | 72.0% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Chichewa | Gemini 3.1 Pro | 89.2% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — Chichewa | GPT-5.4 | 86.7% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — Hausa | Claude Opus 4.7 | 85.7% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Hausa | Claude Opus 4.6 | 77.7% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Hausa | Claude Sonnet 4.6 | 79.0% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Hausa | Gemini 3.1 Pro | 89.9% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — Hausa | GPT-5.4 | 87.6% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — Shona | Claude Opus 4.7 | 85.8% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Shona | Claude Opus 4.6 | 76.3% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Shona | Claude Sonnet 4.6 | 75.7% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Shona | Gemini 3.1 Pro | 90.4% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — Shona | GPT-5.4 | 88.3% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — Kyrgyz | Claude Opus 4.7 | 87.9% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Kyrgyz | Claude Opus 4.6 | 81.9% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Kyrgyz | Claude Sonnet 4.6 | 81.4% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Kyrgyz | Gemini 3.1 Pro | 88.1% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — Kyrgyz | GPT-5.4 | 89.7% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — Amharic | Claude Opus 4.7 | 88.2% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Amharic | Claude Opus 4.6 | 84.0% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Amharic | Claude Sonnet 4.6 | 83.6% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Amharic | Gemini 3.1 Pro | 91.0% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — Amharic | GPT-5.4 | 89.1% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — Swahili | Claude Opus 4.7 | 88.6% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Swahili | Claude Opus 4.6 | 84.6% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Swahili | Claude Sonnet 4.6 | 83.4% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Swahili | Gemini 3.1 Pro | 91.3% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — Swahili | GPT-5.4 | 89.2% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — Sinhala | Claude Opus 4.7 | 89.5% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Sinhala | Claude Opus 4.6 | 85.9% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Sinhala | Claude Sonnet 4.6 | 85.9% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Sinhala | Gemini 3.1 Pro | 92.5% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — Sinhala | GPT-5.4 | 90.6% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — Nepali | Claude Opus 4.7 | 90.1% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Nepali | Claude Opus 4.6 | 87.6% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Nepali | Claude Sonnet 4.6 | 87.3% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Nepali | Gemini 3.1 Pro | 92.6% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — Nepali | GPT-5.4 | 90.9% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — Average (all languages) | Claude Opus 4.7 | 89.9% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Average (all languages) | Claude Opus 4.6 | 86.8% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Average (all languages) | Claude Sonnet 4.6 | 86.1% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Average (all languages) | Gemini 3.1 Pro | 92.2% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — Average (all languages) | GPT-5.4 | 90.6% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — Average gap to English | Claude Opus 4.7 | -3.6% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Average gap to English | Claude Opus 4.6 | -6.1% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Average gap to English | Claude Sonnet 4.6 | -5.9% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Average gap to English | Gemini 3.1 Pro | -2.1% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — Average gap to English | GPT-5.4 | -2.7% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — Worst gap to English | Claude Opus 4.7 | -12.1% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Worst gap to English | Claude Opus 4.6 | -22.6% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Worst gap to English | Claude Sonnet 4.6 | -19.9% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.1 |
| GMMLU — Worst gap to English | Gemini 3.1 Pro | -6.2% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| GMMLU — Worst gap to English | GPT-5.4 | -9.5% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.1 |
| MILU — English | Claude Opus 4.7 | 92.4% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — English | Claude Opus 4.6 | 90.5% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — English | Claude Sonnet 4.6 | 89.7% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — English | Gemini 3.1 Pro | 95.3% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.2 |
| MILU — English | GPT-5.4 | 92.9% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.2 |
| MILU — Malayalam | Claude Opus 4.7 | 87.3% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Malayalam | Claude Opus 4.6 | 85.3% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Malayalam | Claude Sonnet 4.6 | 84.8% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Malayalam | Gemini 3.1 Pro | 91.3% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.2 |
| MILU — Malayalam | GPT-5.4 | 88.1% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.2 |
| MILU — Punjabi | Claude Opus 4.7 | 87.7% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Punjabi | Claude Opus 4.6 | 85.6% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Punjabi | Claude Sonnet 4.6 | 85.2% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Punjabi | Gemini 3.1 Pro | 91.9% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.2 |
| MILU — Punjabi | GPT-5.4 | 88.7% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.2 |
| MILU — Odia | Claude Opus 4.7 | 87.8% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Odia | Claude Opus 4.6 | 85.3% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Odia | Claude Sonnet 4.6 | 84.9% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Odia | Gemini 3.1 Pro | 92.1% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.2 |
| MILU — Odia | GPT-5.4 | 89.6% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.2 |
| MILU — Tamil | Claude Opus 4.7 | 89.0% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Tamil | Claude Opus 4.6 | 85.9% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Tamil | Claude Sonnet 4.6 | 85.1% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Tamil | Gemini 3.1 Pro | 93.7% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.2 |
| MILU — Tamil | GPT-5.4 | 90.5% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.2 |
| MILU — Marathi | Claude Opus 4.7 | 89.5% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Marathi | Claude Opus 4.6 | 86.8% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Marathi | Claude Sonnet 4.6 | 86.3% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Marathi | Gemini 3.1 Pro | 93.0% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.2 |
| MILU — Marathi | GPT-5.4 | 90.2% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.2 |
| MILU — Gujarati | Claude Opus 4.7 | 89.7% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Gujarati | Claude Opus 4.6 | 87.2% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Gujarati | Claude Sonnet 4.6 | 86.5% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Gujarati | Gemini 3.1 Pro | 93.1% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.2 |
| MILU — Gujarati | GPT-5.4 | 89.7% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.2 |
| MILU — Telugu | Claude Opus 4.7 | 90.1% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Telugu | Claude Opus 4.6 | 87.3% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Telugu | Claude Sonnet 4.6 | 86.5% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Telugu | Gemini 3.1 Pro | 93.6% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.2 |
| MILU — Telugu | GPT-5.4 | 90.4% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.2 |
| MILU — Bengali | Claude Opus 4.7 | 91.3% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Bengali | Claude Opus 4.6 | 89.0% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Bengali | Claude Sonnet 4.6 | 88.8% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Bengali | Gemini 3.1 Pro | 94.1% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.2 |
| MILU — Bengali | GPT-5.4 | 91.1% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.2 |
| MILU — Kannada | Claude Opus 4.7 | 91.6% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Kannada | Claude Opus 4.6 | 90.0% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Kannada | Claude Sonnet 4.6 | 89.4% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Kannada | Gemini 3.1 Pro | 94.9% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.2 |
| MILU — Kannada | GPT-5.4 | 91.8% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.2 |
| MILU — Hindi | Claude Opus 4.7 | 93.0% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Hindi | Claude Opus 4.6 | 90.8% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Hindi | Claude Sonnet 4.6 | 90.5% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Hindi | Gemini 3.1 Pro | 96.8% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.2 |
| MILU — Hindi | GPT-5.4 | 93.9% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.2 |
| MILU — Average (all languages) | Claude Opus 4.7 | 89.9% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Average (all languages) | Claude Opus 4.6 | 87.6% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Average (all languages) | Claude Sonnet 4.6 | 87.1% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Average (all languages) | Gemini 3.1 Pro | 93.6% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.2 |
| MILU — Average (all languages) | GPT-5.4 | 90.6% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.2 |
| MILU — Average gap to English | Claude Opus 4.7 | -2.7% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Average gap to English | Claude Opus 4.6 | -3.2% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Average gap to English | Claude Sonnet 4.6 | -2.9% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Average gap to English | Gemini 3.1 Pro | -1.9% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.2 |
| MILU — Average gap to English | GPT-5.4 | -2.5% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.2 |
| MILU — Worst gap to English | Claude Opus 4.7 | -5.1% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Worst gap to English | Claude Opus 4.6 | -5.2% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Worst gap to English | Claude Sonnet 4.6 | -4.9% | structured JSON output, medium extended-thinking budget; accuracy on successfully parsed responses; developer-reported | 8.12.2 |
| MILU — Worst gap to English | Gemini 3.1 Pro | -4.0% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.2 |
| MILU — Worst gap to English | GPT-5.4 | -4.9% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.2 |
| INCLUDE — German | Claude Opus 4.7 | 69.8% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — German | Claude Opus 4.6 | 74.1% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — German | Claude Sonnet 4.6 | 71.2% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — German | Gemini 3.1 Pro | 76.3% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — German | GPT-5.4 | 74.1% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Turkish | Claude Opus 4.7 | 70.2% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Turkish | Claude Opus 4.6 | 70.6% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Turkish | Claude Sonnet 4.6 | 71.2% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Turkish | Gemini 3.1 Pro | 70.9% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Turkish | GPT-5.4 | 71.3% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Urdu | Claude Opus 4.7 | 74.5% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Urdu | Claude Opus 4.6 | 67.0% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Urdu | Claude Sonnet 4.6 | 72.8% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Urdu | Gemini 3.1 Pro | 84.9% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Urdu | GPT-5.4 | 77.3% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Uzbek | Claude Opus 4.7 | 79.1% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Uzbek | Claude Opus 4.6 | 77.9% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Uzbek | Claude Sonnet 4.6 | 79.8% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Uzbek | Gemini 3.1 Pro | 82.4% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Uzbek | GPT-5.4 | 83.8% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Telugu | Claude Opus 4.7 | 79.5% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Telugu | Claude Opus 4.6 | 77.9% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Telugu | Claude Sonnet 4.6 | 77.1% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Telugu | Gemini 3.1 Pro | 87.8% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Telugu | GPT-5.4 | 80.5% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Nepali | Claude Opus 4.7 | 80.1% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Nepali | Claude Opus 4.6 | 78.5% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Nepali | Claude Sonnet 4.6 | 77.0% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Nepali | Gemini 3.1 Pro | 89.8% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Nepali | GPT-5.4 | 85.8% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Basque | Claude Opus 4.7 | 81.9% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Basque | Claude Opus 4.6 | 75.2% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Basque | Claude Sonnet 4.6 | 76.6% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Basque | Gemini 3.1 Pro | 89.8% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Basque | GPT-5.4 | 86.0% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Azerbaijani | Claude Opus 4.7 | 82.2% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Azerbaijani | Claude Opus 4.6 | 80.3% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Azerbaijani | Claude Sonnet 4.6 | 80.0% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Azerbaijani | Gemini 3.1 Pro | 85.4% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Azerbaijani | GPT-5.4 | 83.6% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Hindi | Claude Opus 4.7 | 82.5% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Hindi | Claude Opus 4.6 | 82.1% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Hindi | Claude Sonnet 4.6 | 82.4% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Hindi | Gemini 3.1 Pro | 88.1% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Hindi | GPT-5.4 | 83.8% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Russian | Claude Opus 4.7 | 83.7% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Russian | Claude Opus 4.6 | 83.9% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Russian | Claude Sonnet 4.6 | 83.2% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Russian | Gemini 3.1 Pro | 84.4% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Russian | GPT-5.4 | 84.8% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Malayalam | Claude Opus 4.7 | 84.4% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Malayalam | Claude Opus 4.6 | 83.4% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Malayalam | Claude Sonnet 4.6 | 83.0% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Malayalam | Gemini 3.1 Pro | 90.4% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Malayalam | GPT-5.4 | 87.5% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Finnish | Claude Opus 4.7 | 84.9% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Finnish | Claude Opus 4.6 | 86.4% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Finnish | Claude Sonnet 4.6 | 84.2% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Finnish | Gemini 3.1 Pro | 89.5% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Finnish | GPT-5.4 | 86.9% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Persian | Claude Opus 4.7 | 84.9% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Persian | Claude Opus 4.6 | 84.7% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Persian | Claude Sonnet 4.6 | 79.9% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Persian | Gemini 3.1 Pro | 92.0% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Persian | GPT-5.4 | 84.9% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Armenian | Claude Opus 4.7 | 85.0% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Armenian | Claude Opus 4.6 | 82.1% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Armenian | Claude Sonnet 4.6 | 82.7% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Armenian | Gemini 3.1 Pro | 90.7% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Armenian | GPT-5.4 | 87.7% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Hungarian | Claude Opus 4.7 | 85.3% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Hungarian | Claude Opus 4.6 | 85.1% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Hungarian | Claude Sonnet 4.6 | 86.4% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Hungarian | Gemini 3.1 Pro | 87.3% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Hungarian | GPT-5.4 | 84.9% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Arabic | Claude Opus 4.7 | 85.5% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Arabic | Claude Opus 4.6 | 85.9% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Arabic | Claude Sonnet 4.6 | 84.2% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Arabic | Gemini 3.1 Pro | 90.0% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Arabic | GPT-5.4 | 85.9% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Tamil | Claude Opus 4.7 | 85.6% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Tamil | Claude Opus 4.6 | 83.8% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Tamil | Claude Sonnet 4.6 | 85.1% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Tamil | Gemini 3.1 Pro | 95.5% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Tamil | GPT-5.4 | 90.5% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Portuguese | Claude Opus 4.7 | 85.7% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Portuguese | Claude Opus 4.6 | 86.0% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Portuguese | Claude Sonnet 4.6 | 85.3% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Portuguese | Gemini 3.1 Pro | 87.0% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Portuguese | GPT-5.4 | 84.7% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Kazakh | Claude Opus 4.7 | 85.8% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Kazakh | Claude Opus 4.6 | 82.0% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Kazakh | Claude Sonnet 4.6 | 81.4% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Kazakh | Gemini 3.1 Pro | 94.4% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Kazakh | GPT-5.4 | 90.8% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Indonesian | Claude Opus 4.7 | 86.7% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Indonesian | Claude Opus 4.6 | 84.9% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Indonesian | Claude Sonnet 4.6 | 84.2% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Indonesian | Gemini 3.1 Pro | 88.4% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Indonesian | GPT-5.4 | 85.1% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Korean | Claude Opus 4.7 | 86.8% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Korean | Claude Opus 4.6 | 83.2% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Korean | Claude Sonnet 4.6 | 84.5% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Korean | Gemini 3.1 Pro | 88.4% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Korean | GPT-5.4 | 82.5% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — French | Claude Opus 4.7 | 87.5% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — French | Claude Opus 4.6 | 87.8% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — French | Claude Sonnet 4.6 | 87.8% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — French | Gemini 3.1 Pro | 90.0% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — French | GPT-5.4 | 88.1% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Bengali | Claude Opus 4.7 | 87.6% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Bengali | Claude Opus 4.6 | 87.6% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Bengali | Claude Sonnet 4.6 | 88.3% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Bengali | Gemini 3.1 Pro | 92.5% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Bengali | GPT-5.4 | 88.7% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Ukrainian | Claude Opus 4.7 | 87.9% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Ukrainian | Claude Opus 4.6 | 88.9% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Ukrainian | Claude Sonnet 4.6 | 88.7% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Ukrainian | Gemini 3.1 Pro | 91.5% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Ukrainian | GPT-5.4 | 90.4% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Hebrew | Claude Opus 4.7 | 88.6% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Hebrew | Claude Opus 4.6 | 86.7% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Hebrew | Claude Sonnet 4.6 | 84.0% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Hebrew | Gemini 3.1 Pro | 93.5% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Hebrew | GPT-5.4 | 89.8% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Greek | Claude Opus 4.7 | 89.1% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Greek | Claude Opus 4.6 | 89.7% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Greek | Claude Sonnet 4.6 | 87.5% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Greek | Gemini 3.1 Pro | 91.5% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Greek | GPT-5.4 | 89.7% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Dutch | Claude Opus 4.7 | 89.2% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Dutch | Claude Opus 4.6 | 91.3% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Dutch | Claude Sonnet 4.6 | 89.5% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Dutch | Gemini 3.1 Pro | 90.9% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Dutch | GPT-5.4 | 91.5% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Spanish | Claude Opus 4.7 | 89.5% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Spanish | Claude Opus 4.6 | 89.5% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Spanish | Claude Sonnet 4.6 | 87.6% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Spanish | Gemini 3.1 Pro | 90.2% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Spanish | GPT-5.4 | 87.8% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Belarusian | Claude Opus 4.7 | 89.7% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Belarusian | Claude Opus 4.6 | 86.5% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Belarusian | Claude Sonnet 4.6 | 83.3% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Belarusian | Gemini 3.1 Pro | 93.6% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Belarusian | GPT-5.4 | 92.7% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Tagalog | Claude Opus 4.7 | 89.8% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Tagalog | Claude Opus 4.6 | 89.0% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Tagalog | Claude Sonnet 4.6 | 89.6% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Tagalog | Gemini 3.1 Pro | 91.8% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Tagalog | GPT-5.4 | 90.8% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Polish | Claude Opus 4.7 | 90.0% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Polish | Claude Opus 4.6 | 89.6% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Polish | Claude Sonnet 4.6 | 88.7% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Polish | Gemini 3.1 Pro | 94.3% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Polish | GPT-5.4 | 91.2% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Chinese | Claude Opus 4.7 | 90.9% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Chinese | Claude Opus 4.6 | 91.6% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Chinese | Claude Sonnet 4.6 | 91.6% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Chinese | Gemini 3.1 Pro | 92.7% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Chinese | GPT-5.4 | 90.1% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Albanian | Claude Opus 4.7 | 91.3% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Albanian | Claude Opus 4.6 | 90.9% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Albanian | Claude Sonnet 4.6 | 91.3% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Albanian | Gemini 3.1 Pro | 93.5% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Albanian | GPT-5.4 | 92.0% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Malay | Claude Opus 4.7 | 91.4% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Malay | Claude Opus 4.6 | 89.4% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Malay | Claude Sonnet 4.6 | 88.6% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Malay | Gemini 3.1 Pro | 94.0% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Malay | GPT-5.4 | 92.2% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Georgian | Claude Opus 4.7 | 91.9% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Georgian | Claude Opus 4.6 | 93.4% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Georgian | Claude Sonnet 4.6 | 92.0% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Georgian | Gemini 3.1 Pro | 96.4% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Georgian | GPT-5.4 | 94.2% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Macedonian | Claude Opus 4.7 | 92.9% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Macedonian | Claude Opus 4.6 | 94.0% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Macedonian | Claude Sonnet 4.6 | 93.3% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Macedonian | Gemini 3.1 Pro | 94.7% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Macedonian | GPT-5.4 | 94.4% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Vietnamese | Claude Opus 4.7 | 93.1% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Vietnamese | Claude Opus 4.6 | 90.0% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Vietnamese | Claude Sonnet 4.6 | 88.5% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Vietnamese | Gemini 3.1 Pro | 94.7% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Vietnamese | GPT-5.4 | 91.6% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Bulgarian | Claude Opus 4.7 | 93.5% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Bulgarian | Claude Opus 4.6 | 92.4% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Bulgarian | Claude Sonnet 4.6 | 93.3% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Bulgarian | Gemini 3.1 Pro | 95.8% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Bulgarian | GPT-5.4 | 94.7% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Croatian | Claude Opus 4.7 | 93.6% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Croatian | Claude Opus 4.6 | 93.3% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Croatian | Claude Sonnet 4.6 | 92.4% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Croatian | Gemini 3.1 Pro | 93.8% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Croatian | GPT-5.4 | 93.6% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Italian | Claude Opus 4.7 | 94.0% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Italian | Claude Opus 4.6 | 93.2% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Italian | Claude Sonnet 4.6 | 94.0% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Italian | Gemini 3.1 Pro | 95.1% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Italian | GPT-5.4 | 94.8% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Lithuanian | Claude Opus 4.7 | 94.0% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Lithuanian | Claude Opus 4.6 | 94.4% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Lithuanian | Claude Sonnet 4.6 | 94.2% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Lithuanian | Gemini 3.1 Pro | 95.5% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Lithuanian | GPT-5.4 | 95.2% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Serbian | Claude Opus 4.7 | 95.2% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Serbian | Claude Opus 4.6 | 95.5% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Serbian | Claude Sonnet 4.6 | 95.3% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Serbian | Gemini 3.1 Pro | 96.4% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Serbian | GPT-5.4 | 96.2% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Japanese | Claude Opus 4.7 | 95.9% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Japanese | Claude Opus 4.6 | 96.0% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Japanese | Claude Sonnet 4.6 | 95.0% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Japanese | Gemini 3.1 Pro | 97.4% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Japanese | GPT-5.4 | 96.4% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Estonian | Claude Opus 4.7 | 96.0% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Estonian | Claude Opus 4.6 | 94.6% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Estonian | Claude Sonnet 4.6 | 93.8% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Estonian | Gemini 3.1 Pro | 97.8% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Estonian | GPT-5.4 | 98.2% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Average (all languages) | Claude Opus 4.7 | 87.0% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Average (all languages) | Claude Opus 4.6 | 86.1% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Average (all languages) | Claude Sonnet 4.6 | 85.7% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.12.3 |
| INCLUDE — Average (all languages) | Gemini 3.1 Pro | 90.7% | structured JSON output, dynamic thinking at its default high level; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
| INCLUDE — Average (all languages) | GPT-5.4 | 88.3% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.7 system card | 8.12.3 |
Relationships
- supersedes: Claude Opus 4.6 System Card — Opus 4.7 is the new most-capable generally-accessible Claude model.
- depends-on: Anthropic's Responsible Scaling Policy (Version 3.1) — evaluations keyed to RSP v3.1 thresholds.
- related: Claude Mythos Preview System Card — Mythos Preview is the limited-access frontier; Opus 4.7 compared against it throughout.
- related: Claude Sonnet 4.6 System Card — concurrent Claude 4.6 family model compared on alignment profile.
- related: Anthropic Sabotage Risk Report: Claude Opus 4.6 — Opus 4.6 sabotage risk report context for the alignment-risk pathways discussion.
- related: Emotion Concepts and their Function in a Large Language Model — internal emotion representations used to contextualize the welfare positive-affect finding.
- related: Claude's Constitution — character adherence a headline alignment metric.
- related: UK AISI — Advanced AI Evaluations May Update / UK AISI Frontier AI Trends Report 2025 — UK AISI performed the cyber range external evaluation.
- related: GPT-5.4 Thinking System Card — primary comparator among frontier competitors on GDPval-AA and other capability evaluations.
- related: (Source: anthropic.com) — safeguards methodology context.