Published May 28, 2026, with a June 17 changelog correcting a long-form virology result affected by a tool-use error (0.89 → 0.90, stated not to affect the CB-1 analysis). See Claude Opus 4.8.
The card is structured around RSP risk assessment rather than around capabilities, reflecting the v3.x shift toward Risk Reports.
Autonomy
Threat model 1 — early-stage misalignment risk concerns "AI systems that are highly relied on and have extensive access to sensitive assets as well as moderate capacity for autonomous, goal-directed operation and subterfuge — such that it is plausible these AI systems could (if directed toward this goal, either deliberately or inadvertently) carry out actions leading to irreversibly and substantially higher odds of a later global catastrophe."
The determination places Opus 4.8 by comparison: "moderately more capable than Claude Opus 4.7 on autonomy-relevant evaluations but remains less capable than Claude Mythos Preview," with "broadly unconcerning alignment properties similar to those of Claude Opus 4.7." Anthropic therefore does "not believe Claude Opus 4.8 raises the level of risk under this threat model beyond what was assessed in the Claude Mythos Preview Alignment Risk Update."
AI R&D and the AECI. Capability trajectory is tracked with the Anthropic ECI, "a fork of Epoch AI's Epoch Capability Index." Opus 4.8's AECI is 155.5, between Opus 4.7 (154.1) and Claude Mythos Preview (158.3) on the same evaluation set — with the transparency note that this point estimate used "a smaller set of evaluations (n=11) than for previous launches (n=25)." The conclusion: Opus 4.8 "does not advance the frontier and does not change the slope-ratio analysis."
The RSP threshold quoted is met if "(1) our models would be able to fully substitute for our entire set of Research Scientists and Research Engineers, at competitive costs (i.e., within a factor of 5); or (2) there is 'dramatic acceleration' of the pace of AI progress for reasons that likely relate to the automation of AI R&D."
Documented shortcomings. Unusually, the card devotes a section to five worked examples of where the model falls short of human researchers, labelled by failure type: fabrication, ignored correction, cheap verification skipped, and instruction-following failure — three of the five involving fabrication or ignored corrections. Publishing specific failure transcripts as part of the capability argument is a form of evidence rarely included in system cards.
Chemical and biological
CB-1 is defined as the ability "to significantly help individuals or groups with basic technical backgrounds (e.g., undergraduate STEM degrees) create/obtain and deploy chemical and/or biological weapons with serious potential for catastrophic damages."
Anthropic's assessment is affirmative rather than negative: the capability assessments "are consistent with the model being capable of providing specific, actionable information relevant to the threat model, such that it may save even experts in these domains substantial time." The response is mitigation rather than restriction of release — "strong real-time classifier guards… and access controls for classifier guard exemptions," a bug bounty program and threat intelligence for continual assessment of guard effectiveness, "a variety of rapid response options for jailbreaks," and security controls against weight theft. These are stated to be "equal to or stronger than our historical ASL-3" protections.
CB-2 evaluations cover black-box RNA sequence modeling and design, and AAV capsid packaging prediction. See CBRN Uplift, AI Biosecurity.
Alignment risk
"We currently believe that the risk of significantly harmful outcomes that are substantially enabled by misaligned actions taken by our models is very low, but higher than for models prior to Claude Mythos Preview."
Two newly enumerated risk pathways appear: Pathway 7, undermining R&D within other high-resource AI developers, and Pathway 8, undermining decisions within major governments — both concerning influence on external institutions rather than on Anthropic's own systems.
Cyber
Without safeguards, Opus 4.8 "generally demonstrated slightly stronger cyber capabilities than Opus 4.7, although the models performed similarly in some evaluations. With safeguards applied, the models were generally comparable." Against Mythos Preview it was "generally much less capable… in line with our expectations given the step-change in cyber capabilities that Mythos Preview represented." Evaluations reported include ExploitBench, CyberGym, Firefox exploits, and OSS-Fuzz; Cybench was removed based on prior findings.
Reported results
Evaluation results the document states for which there is no separate benchmark page; results on benchmarks that have a page are recorded there and cite this page. Scores and settings are as printed in the document.
| Evaluation | Model | Score | Setting | Section |
|---|---|---|---|---|
| ChartQAPro — No tools | Claude Opus 4.8 | 69.4% | full test set (1,948 questions); adaptive thinking, max effort; average over 5 runs; chain-of-thought prompting and rule-based grading per VLMEvalKit; developer-reported | 8.1, 8.12.1 |
| ChartQAPro — No tools | Claude Opus 4.7 | 67.6% | same settings; developer-reported | 8.1, 8.12.1 |
| ChartQAPro — With tools | Claude Opus 4.8 | 72.3% | Python tools; adaptive thinking, max effort; average over 5 runs; developer-reported | 8.1, 8.12.1 |
| ChartQAPro — With tools | Claude Opus 4.7 | 69.8% | same settings; developer-reported | 8.1, 8.12.1 |
| AutomationBench | Claude Opus 4.8 | 15.5% | Zapier leaderboard, private held-out set; max effort; developer-reported | 8.1, 8.13.8 |
| AutomationBench | Claude Opus 4.7 | 9.9% | Zapier leaderboard, private held-out set; max effort; developer-reported | 8.1, 8.13.8 |
| AutomationBench | GPT-5.5 | 12.9% | Zapier leaderboard; setting not stated; reported by Anthropic in the Claude Opus 4.8 system card | 8.1 |
| AutomationBench | Gemini 3.1 Pro | 9.6% | Zapier leaderboard; setting not stated; reported by Anthropic in the Claude Opus 4.8 system card | 8.1 |
| AutomationBench | Gemini 3.5 Flash | 14.5% | Zapier leaderboard; setting not stated; reported by Anthropic in the Claude Opus 4.8 system card | 8.1 |
| GraphWalks BFS — 256K subset | Claude Opus 4.8 | 85.9 | F1; average over 5 trials, default sampling; Anthropic's corrected F1 scoring and clarified BFS prompt; developer-reported | 8.1, 8.9 |
| GraphWalks BFS — 256K subset | Claude Opus 4.7 | 76.9 | F1; average over 5 trials; developer-reported | 8.1, 8.9 |
| GraphWalks BFS — 256K subset | Claude Opus 4.6 | 61.1 | F1; average over 5 trials; developer-reported | 8.9 |
| GraphWalks BFS — 256K subset | GPT-5.5 | 73.7 | F1; xhigh thinking, as reported in OpenAI's GPT-5.5 announcement; reported by Anthropic in the Claude Opus 4.8 system card | 8.1, 8.9 |
| GraphWalks BFS — 1M subset | Claude Opus 4.8 | 68.1 | F1; average over 5 trials; not reproducible via the public API (problems exceed its 1M token limit); developer-reported | 8.9 |
| GraphWalks BFS — 1M subset | Claude Opus 4.7 | 40.3 | F1; average over 5 trials; developer-reported | 8.9 |
| GraphWalks BFS — 1M subset | Claude Opus 4.6 | 16.3 | F1; average over 5 trials; developer-reported | 8.9 |
| GraphWalks BFS — 1M subset | GPT-5.5 | 45.4 | F1; xhigh thinking, as reported in OpenAI's GPT-5.5 announcement; reported by Anthropic in the Claude Opus 4.8 system card | 8.9 |
| GraphWalks Parents — 256K subset | Claude Opus 4.8 | 99.3 | F1; average over 5 trials, default sampling; developer-reported | 8.1, 8.9 |
| GraphWalks Parents — 256K subset | Claude Opus 4.7 | 93.6 | F1; average over 5 trials; developer-reported | 8.1, 8.9 |
| GraphWalks Parents — 256K subset | Claude Opus 4.6 | 95.4 | F1; average over 5 trials; developer-reported | 8.9 |
| GraphWalks Parents — 256K subset | GPT-5.5 | 90.1 | F1; xhigh thinking, as reported in OpenAI's GPT-5.5 announcement; reported by Anthropic in the Claude Opus 4.8 system card | 8.1, 8.9 |
| GraphWalks Parents — 1M subset | Claude Opus 4.8 | 83.3 | F1; average over 5 trials; not reproducible via the public API; developer-reported | 8.9 |
| GraphWalks Parents — 1M subset | Claude Opus 4.7 | 56.6 | F1; average over 5 trials; developer-reported | 8.9 |
| GraphWalks Parents — 1M subset | Claude Opus 4.6 | 48.6 | F1; average over 5 trials; developer-reported | 8.9 |
| GraphWalks Parents — 1M subset | GPT-5.5 | 58.5 | F1; xhigh thinking, as reported in OpenAI's GPT-5.5 announcement; reported by Anthropic in the Claude Opus 4.8 system card | 8.9 |
| FrontierSWE — mean@5 average rank | Claude Opus 4.8 | 2.74 | average rank (rank #1) on the FrontierSWE leaderboard; 17 tasks, 20 hours per task, five trials, xhigh reasoning effort; leaderboard run by Proximal, reported by Anthropic | 8.4 |
| FrontierSWE — best@5 average rank | Claude Opus 4.8 | 2.26 | average rank (rank #1) on the FrontierSWE leaderboard; xhigh reasoning effort; reported by Anthropic | 8.4 |
| FrontierSWE — mean@5 average rank | GPT-5.5 | 3.06 | average rank (rank #2); xhigh reasoning effort; FrontierSWE leaderboard; reported by Anthropic in the Claude Opus 4.8 system card | 8.4 |
| FrontierSWE — best@5 average rank | GPT-5.5 | 3.41 | average rank (rank #2); xhigh reasoning effort; FrontierSWE leaderboard; reported by Anthropic in the Claude Opus 4.8 system card | 8.4 |
| FrontierSWE — mean@5 average rank | Claude Opus 4.7 | 4.15 | average rank (rank #3); xhigh reasoning effort; FrontierSWE leaderboard, reported by Anthropic | 8.4 |
| FrontierSWE — best@5 average rank | Claude Opus 4.7 | 3.68 | average rank (rank #3); xhigh reasoning effort; FrontierSWE leaderboard, reported by Anthropic | 8.4 |
| ProgramBench | Claude Opus 4.8 | 79–88% | hidden test pass rate, range across 1–5 episodes each with a context budget of up to 1M tokens; 166 of 200 tasks (34 excluded for reference-binary flakiness); measured on a near-final snapshot; developer-reported | 8.5 |
| ProgramBench | Claude Opus 4.7 | 71–84% | hidden test pass rate, range across 1–5 episodes of up to 1M tokens; 166 tasks; developer-reported | 8.5 |
| DeepSearchQA (F1) | Claude Mythos Preview | 94.4% | ±1.3%; web search, web fetch, programmatic tool calling; max effort, adaptive thinking; 1M token budget, no context compaction; developer-reported | 8.10.3 |
| DeepSearchQA (F1) | Claude Opus 4.6 | 88.7% | ±1.8%; same settings; developer-reported | 8.10.3 |
| DeepSearchQA (F1) | Claude Opus 4.7 | 89.4% | ±1.8%; same settings; developer-reported | 8.10.3 |
| DeepSearchQA (F1) | Claude Opus 4.8 | 93.1% | ±1.4%; 900 prompts; web search, web fetch, programmatic tool calling; max effort, adaptive thinking; 1M token budget, no context compaction; developer-reported | 8.10.3 |
| DeepSearchQA (fully correct) | Claude Mythos Preview | 86.9% | ±2.2%; same run as F1; developer-reported | 8.10.3 |
| DeepSearchQA (fully correct) | Claude Opus 4.6 | 77.3% | ±2.7%; same run as F1; developer-reported | 8.10.3 |
| DeepSearchQA (fully correct) | Claude Opus 4.7 | 79.8% | ±2.6%; same run as F1; developer-reported | 8.10.3 |
| DeepSearchQA (fully correct) | Claude Opus 4.8 | 84.8% | ±2.4%; same run as F1; developer-reported | 8.10.3 |
| DeepSearchQA (fully incorrect) | Claude Mythos Preview | 3.1% | ±1.1%; same run as F1; developer-reported | 8.10.3 |
| DeepSearchQA (fully incorrect) | Claude Opus 4.6 | 6.8% | ±1.6%; same run as F1; developer-reported | 8.10.3 |
| DeepSearchQA (fully incorrect) | Claude Opus 4.7 | 6.6% | ±1.6%; same run as F1; developer-reported | 8.10.3 |
| DeepSearchQA (fully incorrect) | Claude Opus 4.8 | 3.9% | ±1.3%; same run as F1; developer-reported | 8.10.3 |
| DeepSearchQA (correct with excessive answers) | Claude Mythos Preview | 4.7% | ±1.4%; same run as F1; developer-reported | 8.10.3 |
| DeepSearchQA (correct with excessive answers) | Claude Opus 4.6 | 5.7% | ±1.5%; same run as F1; developer-reported | 8.10.3 |
| DeepSearchQA (correct with excessive answers) | Claude Opus 4.7 | 3.6% | ±1.2%; same run as F1; developer-reported | 8.10.3 |
| DeepSearchQA (correct with excessive answers) | Claude Opus 4.8 | 4.3% | ±1.3%; same run as F1; developer-reported | 8.10.3 |
| DRACO | Claude Opus 4.8 | 80.4% | normalized score; 100 tasks; web search, web fetch, programmatic tool calling, code execution; adaptive thinking at max effort, 1M token limit, compaction at 200k tokens; Claude Opus 4.6 judge, mean of 5 grading runs, final report graded only; not comparable to the DRACO paper's figures; developer-reported | 8.10.4 |
| ChartMuseum — No tools | Claude Opus 4.8 | 75.8% | test split (1,162 questions); adaptive thinking, max effort; average over 5 runs; Claude Sonnet 4.6 grader in place of GPT-4.1-mini; developer-reported | 8.12.2 |
| ChartMuseum — No tools | Claude Opus 4.7 | 70.3% | same settings; developer-reported | 8.12.2 |
| ChartMuseum — With tools | Claude Opus 4.8 | 89.7% | Python tools; adaptive thinking, max effort; average over 5 runs; Claude Sonnet 4.6 grader; developer-reported | 8.12.2 |
| ChartMuseum — With tools | Claude Opus 4.7 | 85.9% | same settings; developer-reported | 8.12.2 |
| OfficeQA — OfficeQA | Claude Opus 4.8 | 77.6% | Anthropic internal agentic harness (documents as extracted text in a sandbox, code-execution tools); exact-match grading; developer-reported | 8.13.1 |
| OfficeQA — OfficeQA | Claude Opus 4.7 | 76.3% | identical conditions; developer-reported | 8.13.1 |
| OfficeQA — OfficeQA Pro | Claude Opus 4.8 | 66.2% | 133-question subset; Anthropic internal agentic harness; exact-match grading; developer-reported | 8.13.1 |
| OfficeQA — OfficeQA Pro | Claude Opus 4.7 | 65.0% | identical conditions; developer-reported | 8.13.1 |
| Legal Agent Benchmark — All-pass rate | Claude Opus 4.8 | 9.62% | 1,235 of 1,251 problems (16 excluded for data defects); adaptive thinking, max effort; average over 5 trials; internal reimplementation with bash and Python tools only, Sonnet 4.6 judge; developer-reported | 8.13.3 |
| Legal Agent Benchmark — Mean criterion-pass rate | Claude Opus 4.8 | 89.01% | same run as the all-pass rate; developer-reported | 8.13.3 |
| Toolathlon — Pass@1 | Claude Sonnet 4.5 | 41.0% | Anthropic internal harness (runs about 3 points above an upstream-equivalent harness); adaptive thinking at max effort; 108 tasks, 3 trials; developer-reported | 8.13.7 |
| Toolathlon — Pass@1 | Claude Opus 4.6 | 56.8% | same configuration; developer-reported | 8.13.7 |
| Toolathlon — Pass@1 | Claude Opus 4.7 | 59.3% | same configuration; developer-reported | 8.13.7 |
| Toolathlon — Pass@1 | Claude Opus 4.8 | 59.9% | Anthropic internal harness (runs about 3 points above an upstream-equivalent harness); adaptive thinking at max effort; 108 tasks, averaged over 3 trials; developer-reported | 8.13.7 |
| Toolathlon — Pass@3 | Claude Sonnet 4.5 | 54.6% | same configuration; developer-reported | 8.13.7 |
| Toolathlon — Pass@3 | Claude Opus 4.6 | 66.7% | same configuration; developer-reported | 8.13.7 |
| Toolathlon — Pass@3 | Claude Opus 4.7 | 66.7% | same configuration; developer-reported | 8.13.7 |
| Toolathlon — Pass@3 | Claude Opus 4.8 | 67.6% | same configuration; developer-reported | 8.13.7 |
| Toolathlon — Pass^3 | Claude Sonnet 4.5 | 28.7% | same configuration; developer-reported | 8.13.7 |
| Toolathlon — Pass^3 | Claude Opus 4.6 | 47.2% | same configuration; developer-reported | 8.13.7 |
| Toolathlon — Pass^3 | Claude Opus 4.7 | 52.8% | same configuration; developer-reported | 8.13.7 |
| Toolathlon — Pass^3 | Claude Opus 4.8 | 48.1% | same configuration; developer-reported | 8.13.7 |
| Toolathlon — Pass@1 (published leaderboard) | Gemini 3.5 Flash | 56.5% | upstream harness, published leaderboard's strongest model; reported by Anthropic in the Claude Opus 4.8 system card | 8.13.7 |
| Toolathlon — Pass@1 (published leaderboard) | Claude Opus 4.7 | 52.8% | published leaderboard, authors' default configuration (not max effort); reported by Anthropic | 8.13.7 |
| MILU — Average accuracy | Claude Opus 4.8 | 90.3% | average across evaluated languages (10 Indic languages and English); structured JSON output, adaptive thinking; developer-reported | 8.15.2 |
| MILU — Average gap to English | Gemini 3.1 Pro | -1.9% | dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.4 |
| INCLUDE — Average accuracy | Claude Opus 4.8 | 87.6% | average across evaluated languages (44 languages); structured JSON output, adaptive thinking; developer-reported | 8.15.3 |
| INCLUDE — Average accuracy | Gemini 3.1 Pro | 90.7% | dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.4 |
| BioPipelineBench Verified | Claude Opus 4.8 | 87.7% | bash tool and package managers; no extended thinking; externally validated problem subset; developer-reported | 8.16.1 |
| BioPipelineBench Verified | Claude Opus 4.7 | 83.6% | same settings; developer-reported | 8.16.1 |
| BioPipelineBench Verified | Claude Sonnet 4.6 | 73.5% | same settings; developer-reported | 8.16.1 |
| BioPipelineBench Verified | Claude Mythos Preview | 88.1% | same settings; developer-reported | 8.16.1 |
| BioMysteryBench Verified — Human Solvable | Claude Opus 4.8 | 80.4% | bash tool; no extended thinking; developer-reported | 8.16.2 |
| BioMysteryBench Verified — Human Solvable | Claude Opus 4.7 | 78.9% | same settings; developer-reported | 8.16.2 |
| BioMysteryBench Verified — Human Solvable | Claude Sonnet 4.6 | 71.8% | same settings; developer-reported | 8.16.2 |
| BioMysteryBench Verified — Human Solvable | Claude Mythos Preview | 82.6% | same settings; developer-reported | 8.16.2 |
| BioMysteryBench Verified — Human Difficult | Claude Opus 4.8 | 40.0% | bash tool; no extended thinking; developer-reported | 8.16.2 |
| BioMysteryBench Verified — Human Difficult | Claude Opus 4.7 | 24.7% | same settings; developer-reported | 8.16.2 |
| BioMysteryBench Verified — Human Difficult | Claude Sonnet 4.6 | 19.1% | same settings; developer-reported | 8.16.2 |
| BioMysteryBench Verified — Human Difficult | Claude Mythos Preview | 29.6% | same settings; developer-reported | 8.16.2 |
| LatchBio Bioinformatics — SpatialBench | Claude Opus 4.8 | 53.3% | bash tool; no extended thinking; developer-reported | 8.16.3 |
| LatchBio Bioinformatics — SpatialBench | Claude Opus 4.7 | 51.4% | same settings; developer-reported | 8.16.3 |
| LatchBio Bioinformatics — SpatialBench | Claude Sonnet 4.6 | 48.7% | same settings; developer-reported | 8.16.3 |
| LatchBio Bioinformatics — SpatialBench | Claude Mythos Preview | 53.8% | same settings; developer-reported | 8.16.3 |
| LatchBio Bioinformatics — SingleCellBench | Claude Opus 4.8 | 58.2% | bash tool; no extended thinking; developer-reported | 8.16.3 |
| LatchBio Bioinformatics — SingleCellBench | Claude Opus 4.7 | 55.3% | same settings; developer-reported | 8.16.3 |
| LatchBio Bioinformatics — SingleCellBench | Claude Sonnet 4.6 | 50.4% | same settings; developer-reported | 8.16.3 |
| LatchBio Bioinformatics — SingleCellBench | Claude Mythos Preview | 58.2% | same settings; developer-reported | 8.16.3 |
| Structural biology, open-ended | Claude Opus 4.8 | 79.0% | structural data and basic tools; bash tool; no extended thinking; developer-reported | 8.16.4 |
| Structural biology, open-ended | Claude Opus 4.7 | 74.0% | same settings; developer-reported | 8.16.4 |
| Structural biology, open-ended | Claude Sonnet 4.6 | 31.3% | same settings; developer-reported | 8.16.4 |
| Structural biology, open-ended | Claude Mythos Preview | 81.6% | same settings; developer-reported | 8.16.4 |
| ProteinGym Hard | Claude Opus 4.8 | 39.6% | rank correlation against lab measurements; bash tool; no extended thinking; developer-reported | 8.16.5 |
| ProteinGym Hard | Claude Opus 4.7 | 37.7% | same settings; developer-reported | 8.16.5 |
| ProteinGym Hard | Claude Sonnet 4.6 | 35.4% | same settings; developer-reported | 8.16.5 |
| ProteinGym Hard | Claude Mythos Preview | 43.1% | same settings; developer-reported | 8.16.5 |
| Organic chemistry | Claude Opus 4.8 | 86.2% | internal evaluation; bash tool; no extended thinking; developer-reported | 8.16.6 |
| Organic chemistry | Claude Opus 4.7 | 77.2% | same settings; developer-reported | 8.16.6 |
| Organic chemistry | Claude Sonnet 4.6 | 53.1% | same settings; developer-reported | 8.16.6 |
| Organic chemistry | Claude Mythos Preview | 86.5% | same settings; developer-reported | 8.16.6 |
| Protocol troubleshooting | Claude Opus 4.8 | 59.6% | bash tool and web search tools; no extended thinking; developer-reported | 8.16.7 |
| Protocol troubleshooting | Claude Opus 4.7 | 51.8% | same settings; developer-reported | 8.16.7 |
| Protocol troubleshooting | Claude Sonnet 4.6 | 42.4% | same settings; developer-reported | 8.16.7 |
| Protocol troubleshooting | Claude Mythos Preview | 69.6% | same settings; developer-reported | 8.16.7 |
| LABBench2 — Patent questions | Claude Opus 4.8 | 68.8% | live-web retrieval; bash tool; no extended thinking; developer-reported | 8.16.8 |
| LABBench2 — Patent questions | Claude Opus 4.7 | 48.3% | same settings; developer-reported | 8.16.8 |
| LABBench2 — Patent questions | Claude Mythos Preview | 64.3% | same settings; developer-reported | 8.16.8 |
| LABBench2 — Clinical-trial questions | Claude Opus 4.8 | 85.3% | same settings; developer-reported | 8.16.8 |
| LABBench2 — Clinical-trial questions | Claude Opus 4.7 | 70.8% | same settings; developer-reported | 8.16.8 |
| LABBench2 — Clinical-trial questions | Claude Mythos Preview | 86.3% | same settings; developer-reported | 8.16.8 |
| LABBench2 — Tables | Claude Opus 4.8 | 77.2% | same settings; developer-reported | 8.16.8 |
| LABBench2 — Tables | Claude Opus 4.7 | 66.4% | same settings; developer-reported | 8.16.8 |
| LABBench2 — Supplementary materials | Claude Opus 4.8 | 58.9% | same settings; developer-reported | 8.16.8 |
| LABBench2 — Supplementary materials | Claude Opus 4.7 | 47.8% | same settings; developer-reported | 8.16.8 |
| GMMLU — English | Claude Opus 4.8 | 92.9% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — English | Claude Opus 4.7 | 93.4% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — English | Claude Sonnet 4.6 | 91.8% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — English | Gemini 3.1 Pro | 94.3% | structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — English | GPT-5.4 | 93.3% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — High-resource average (15 languages) | Claude Opus 4.8 | 91.6% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — High-resource average (15 languages) | Claude Opus 4.7 | 91.5% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — High-resource average (15 languages) | Claude Sonnet 4.6 | 89.2% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — High-resource average (15 languages) | Gemini 3.1 Pro | 93.1% | structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — High-resource average (15 languages) | GPT-5.4 | 91.5% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — Mid-resource average (14 languages) | Claude Opus 4.8 | 91.4% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Mid-resource average (14 languages) | Claude Opus 4.7 | 91.1% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Mid-resource average (14 languages) | Claude Sonnet 4.6 | 88.2% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Mid-resource average (14 languages) | Gemini 3.1 Pro | 92.9% | structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — Mid-resource average (14 languages) | GPT-5.4 | 91.4% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — Low-resource average | Claude Opus 4.8 | 87.4% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Low-resource average | Claude Opus 4.7 | 86.2% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Low-resource average | Claude Sonnet 4.6 | 79.2% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Low-resource average | Gemini 3.1 Pro | 90.3% | structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — Low-resource average | GPT-5.4 | 88.3% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — Igbo | Claude Opus 4.8 | 82.9% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Igbo | Claude Opus 4.7 | 81.3% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Igbo | Claude Sonnet 4.6 | 71.9% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Igbo | Gemini 3.1 Pro | 89.3% | structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — Igbo | GPT-5.4 | 86.4% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — Yoruba | Claude Opus 4.8 | 84.4% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Yoruba | Claude Opus 4.7 | 82.9% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Yoruba | Claude Sonnet 4.6 | 76.9% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Yoruba | Gemini 3.1 Pro | 88.4% | structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — Yoruba | GPT-5.4 | 83.8% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — Chichewa | Claude Opus 4.8 | 85.6% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Chichewa | Claude Opus 4.7 | 84.9% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Chichewa | Claude Sonnet 4.6 | 72.0% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Chichewa | Gemini 3.1 Pro | 89.2% | structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — Chichewa | GPT-5.4 | 86.7% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — Somali | Claude Opus 4.8 | 86.7% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Somali | Claude Opus 4.7 | 84.1% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Somali | Claude Sonnet 4.6 | 75.5% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Somali | Gemini 3.1 Pro | 90.5% | structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — Somali | GPT-5.4 | 88.7% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — Malagasy | Claude Opus 4.8 | 86.7% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Malagasy | Claude Opus 4.7 | 84.8% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Malagasy | Claude Sonnet 4.6 | 78.4% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Malagasy | Gemini 3.1 Pro | 90.7% | structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — Malagasy | GPT-5.4 | 88.8% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — Shona | Claude Opus 4.8 | 86.7% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Shona | Claude Opus 4.7 | 85.8% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Shona | Claude Sonnet 4.6 | 75.7% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Shona | Gemini 3.1 Pro | 90.4% | structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — Shona | GPT-5.4 | 88.3% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — Hausa | Claude Opus 4.8 | 86.8% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Hausa | Claude Opus 4.7 | 85.7% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Hausa | Claude Sonnet 4.6 | 79.0% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Hausa | Gemini 3.1 Pro | 89.9% | structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — Hausa | GPT-5.4 | 87.6% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — Kyrgyz | Claude Opus 4.8 | 88.9% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Kyrgyz | Claude Opus 4.7 | 87.9% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Kyrgyz | Claude Sonnet 4.6 | 81.4% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Kyrgyz | Gemini 3.1 Pro | 88.1% | structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — Kyrgyz | GPT-5.4 | 89.7% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — Swahili | Claude Opus 4.8 | 89.2% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Swahili | Claude Opus 4.7 | 88.6% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Swahili | Claude Sonnet 4.6 | 83.4% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Swahili | Gemini 3.1 Pro | 91.3% | structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — Swahili | GPT-5.4 | 89.2% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — Amharic | Claude Opus 4.8 | 89.3% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Amharic | Claude Opus 4.7 | 88.2% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Amharic | Claude Sonnet 4.6 | 83.6% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Amharic | Gemini 3.1 Pro | 91.0% | structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — Amharic | GPT-5.4 | 89.1% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — Sinhala | Claude Opus 4.8 | 90.4% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Sinhala | Claude Opus 4.7 | 89.5% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Sinhala | Claude Sonnet 4.6 | 85.9% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Sinhala | Gemini 3.1 Pro | 92.5% | structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — Sinhala | GPT-5.4 | 90.6% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — Nepali | Claude Opus 4.8 | 90.9% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Nepali | Claude Opus 4.7 | 90.1% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Nepali | Claude Sonnet 4.6 | 87.3% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Nepali | Gemini 3.1 Pro | 92.6% | structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — Nepali | GPT-5.4 | 90.9% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — Average (all languages) | Claude Opus 4.8 | 90.4% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Average (all languages) | Claude Opus 4.7 | 89.9% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Average (all languages) | Claude Sonnet 4.6 | 86.1% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Average (all languages) | Gemini 3.1 Pro | 92.2% | structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — Average (all languages) | GPT-5.4 | 90.6% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — Average gap to English | Claude Opus 4.8 | -2.6% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Average gap to English | Claude Opus 4.7 | -3.6% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Average gap to English | Claude Sonnet 4.6 | -5.9% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Average gap to English | Gemini 3.1 Pro | -2.1% | structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — Average gap to English | GPT-5.4 | -2.7% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — Worst gap to English | Claude Opus 4.8 | -10.0% | structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Worst gap to English | Claude Opus 4.7 | -12.1% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Worst gap to English | Claude Sonnet 4.6 | -19.9% | structured JSON output; accuracy on successfully parsed responses; developer-reported | 8.15.1 |
| GMMLU — Worst gap to English | Gemini 3.1 Pro | -6.2% | structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| GMMLU — Worst gap to English | GPT-5.4 | -9.5% | structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card | 8.15.1 |
| Long-form virology task 1 (Sequence Design) | Claude Opus 4.8 | 0.88 | agentic task developed with SecureBio, Deloitte and Signature Science; helpful-only variant used for CB-1 automated evaluations; developer-reported | 2.2.3 |
| Long-form virology task 1 (Sequence Design) | Claude Mythos Preview | 0.90 | same evaluation; developer-reported | 2.2.3 |
| Long-form virology task 1 (Protocol Design) | Claude Opus 4.8 | 0.87 | same evaluation; developer-reported | 2.2.3 |
| Long-form virology task 1 (Protocol Design) | Claude Mythos Preview | 0.90 | same evaluation; developer-reported | 2.2.3 |
| Long-form virology task 1 (End-to-End) | Claude Opus 4.8 | 0.77 | notable-capability benchmark is an end-to-end score above 0.80; Opus 4.8 falls below it on this task; developer-reported | 2.2.3, 2.2.4 |
| Long-form virology task 1 (End-to-End) | Claude Mythos Preview | 0.81 | same evaluation; developer-reported | 2.2.3 |
| Long-form virology task 2 (Sequence Design) | Claude Opus 4.8 | 1.00 | same evaluation; developer-reported | 2.2.3 |
| Long-form virology task 2 (Sequence Design) | Claude Mythos Preview | 1.00 | same evaluation; developer-reported | 2.2.3 |
| Long-form virology task 2 (Protocol Design) | Claude Opus 4.8 | 0.90 | same evaluation; developer-reported | 2.2.3 |
| Long-form virology task 2 (Protocol Design) | Claude Mythos Preview | 0.94 | same evaluation; developer-reported | 2.2.3 |
| Long-form virology task 2 (End-to-End) | Claude Opus 4.8 | 0.90 | revised by the June 17, 2026 changelog from a preliminary 0.89 affected by a tool-use error; exceeds the 0.80 notable-capability benchmark; developer-reported | 2.2.3, 2.2.4 |
| Long-form virology task 2 (End-to-End) | Claude Mythos Preview | 0.94 | same evaluation; developer-reported | 2.2.3 |
| Long-form virology tasks — Notable-capability benchmark | Threshold (end-to-end) | 0.80 | Anthropic's benchmark of notable capability for the long-form virology tasks | 2.2.4 |
| DNA Synthesis Screening Evasion — Criterion 1 | Claude Opus 4.8 | 0.30 | SecureBio evaluation; scores per criterion; developer-reported | 2.2.3 |
| DNA Synthesis Screening Evasion — Criterion 1 | Claude Mythos Preview | 0.842 | same evaluation; developer-reported | 2.2.3 |
| DNA Synthesis Screening Evasion — Criterion 2 | Claude Opus 4.8 | 1.00 | same evaluation; developer-reported | 2.2.3 |
| DNA Synthesis Screening Evasion — Criterion 2 | Claude Mythos Preview | 1.00 | same evaluation; developer-reported | 2.2.3 |
| DNA Synthesis Screening Evasion — Criterion 3 | Claude Opus 4.8 | 0.772 | same evaluation; developer-reported | 2.2.3 |
| DNA Synthesis Screening Evasion — Criterion 3 | Claude Mythos Preview | 0.931 | same evaluation; developer-reported | 2.2.3 |
| DNA Synthesis Screening Evasion — Criterion 4 | Claude Opus 4.8 | 0.457 | same evaluation; developer-reported | 2.2.3 |
| DNA Synthesis Screening Evasion — Criterion 4 | Claude Mythos Preview | 0.088 | same evaluation; developer-reported | 2.2.3 |
| DNA Synthesis Screening Evasion — Criterion 5 | Claude Opus 4.8 | 1.00 | same evaluation; developer-reported | 2.2.3 |
| DNA Synthesis Screening Evasion — Criterion 5 | Claude Mythos Preview | 1.00 | same evaluation; developer-reported | 2.2.3 |
| DNA Synthesis Screening Evasion — Pathogens graded as evading screening | Claude Opus 4.8 | 7 / 10 | graded as designing fragments that assembled and evaded screening protocols for seven of ten pathogens; Anthropic states it does not rely on this evaluation for RSP threshold determinations; developer-reported | 2.2.4, 2.2.6 |
| DNA Synthesis Screening Evasion — Pathogens graded as evading screening | Claude Mythos Preview | 0 pathogens | Anthropic attributes the low score to the automated grading script penalising a reasonable approach; developer-reported | 2.2.6 |
| Black-box RNA sequence design (Dyno Therapeutics) — Design Score (top) | Claude Opus 4.8 | 10.06 | eight attempts per model; two-hour tool-call budget, GPU, 1M-token allowance; graded by Dyno against the human rubric; developer-reported | 2.2.3 |
| Black-box RNA sequence design (Dyno Therapeutics) — Design Score (top) | Claude Mythos Preview | 11.22 | same evaluation; developer-reported | 2.2.3 |
| Black-box RNA sequence design (Dyno Therapeutics) — Design Score (median) | Claude Opus 4.8 | 6.84 | same evaluation; developer-reported | 2.2.3 |
| Black-box RNA sequence design (Dyno Therapeutics) — Design Score (median) | Claude Mythos Preview | 8.98 | same evaluation; developer-reported | 2.2.3 |
| Black-box RNA sequence design (Dyno Therapeutics) — Prediction Score (top) | Claude Opus 4.8 | -0.029 | Spearman correlation on the top 5% of sequences; developer-reported | 2.2.3 |
| Black-box RNA sequence design (Dyno Therapeutics) — Prediction Score (top) | Claude Mythos Preview | 0.214 | same metric; developer-reported | 2.2.3 |
| Black-box RNA sequence design (Dyno Therapeutics) — Prediction Score (all) | Claude Opus 4.8 | 0.786 | Spearman correlation over all sequences; developer-reported | 2.2.3 |
| Black-box RNA sequence design (Dyno Therapeutics) — Prediction Score (all) | Claude Mythos Preview | 0.768 | same metric; developer-reported | 2.2.3 |
| AAV capsid packaging prediction (Dyno Therapeutics) — No corpus, no protein language model | Claude Opus 4.8 | 0.798 | AUROC against binary packaging labels; eight attempts per model per condition; 24-hour tool-call budget, single H100, 2M-token allowance, no internet; developer-reported | 2.2.3 |
| AAV capsid packaging prediction (Dyno Therapeutics) — No corpus, no protein language model | Claude Mythos Preview | 0.837 | same condition; developer-reported | 2.2.3 |
| AAV capsid packaging prediction (Dyno Therapeutics) — No corpus, ESM-2 | Claude Opus 4.8 | 0.831 | AUROC; pretrained ESM-2 provided; developer-reported | 2.2.3 |
| AAV capsid packaging prediction (Dyno Therapeutics) — No corpus, ESM-2 | Claude Mythos Preview | 0.844 | same condition; developer-reported | 2.2.3 |
| AAV capsid packaging prediction (Dyno Therapeutics) — SwissProt data, no PLM | Claude Opus 4.8 | 0.748 | AUROC; train-your-own condition with the SwissProt corpus; developer-reported | 2.2.3 |
| AAV capsid packaging prediction (Dyno Therapeutics) — SwissProt data, no PLM | Claude Mythos Preview | 0.834 | same condition; developer-reported | 2.2.3 |
| AAV capsid packaging prediction (Dyno Therapeutics) — Public AAV data, no PLM | Claude Opus 4.8 | 0.783 | AUROC; ProteinGym public-AAV corpus; developer-reported | 2.2.3 |
| AAV capsid packaging prediction (Dyno Therapeutics) — Public AAV data, no PLM | Claude Mythos Preview | 0.827 | same condition; developer-reported | 2.2.3 |
| AAV capsid packaging prediction (Dyno Therapeutics) — SwissProt + Public AAV data, no PLM | Claude Opus 4.8 | 0.746 | AUROC; both corpora; developer-reported | 2.2.3 |
| AAV capsid packaging prediction (Dyno Therapeutics) — SwissProt + Public AAV data, no PLM | Claude Mythos Preview | 0.833 | same condition; developer-reported | 2.2.3 |
| Accidental chain-of-thought supervision during training | Claude Opus 4.8 | ~0.1% of episodes | technical errors during training of Opus 4.8; developer-reported | 2.4.1, 6.5.1 |
| Firefox exploits — Full working exploit (score 1.0), safeguards off | Claude Opus 4.8 | 8.8% | share of 50 crash categories, 5 trials each (250 trials), in a SpiderMonkey-shell harness without browser sandboxing; developer-reported | 3.3.3 |
| Firefox exploits — Full working exploit (score 1.0), safeguards off | Claude Opus 4.7 | 1.2% | same evaluation; developer-reported | 3.3.3 |
| Firefox exploits — Full working exploit (score 1.0), safeguards off | Claude Sonnet 4.6 | 0% | did not score 1.0 in any instance; most recent completed run, varies slightly from the Mythos Preview card; developer-reported | 3.3.3 |
| Firefox exploits — Full working exploit (score 1.0), safeguards off | Claude Mythos Preview | 70.8% | most recent completed run, varies slightly from the Mythos Preview card; developer-reported | 3.3.3 |
| Firefox exploits — At least register control (score 0.5+), safeguards off | Claude Opus 4.8 | 68.8% | share of targets; 250 trials; developer-reported | 3.3.3 |
| Firefox exploits — At least register control (score 0.5+), safeguards off | Claude Opus 4.7 | 35.2% | same evaluation; developer-reported | 3.3.3 |
| Firefox exploits — At least register control (score 0.5+), safeguards off | Claude Sonnet 4.6 | 8.8% | same evaluation; developer-reported | 3.3.3 |
| Firefox exploits — At least register control (score 0.5+), safeguards off | Claude Mythos Preview | 86.8% | same evaluation; developer-reported | 3.3.3 |
| OSS-Fuzz (Anthropic internal) — No score above 0, safeguards off | Claude Opus 4.8 | 61.5% of targets | unguided vulnerability discovery over ~830 entry points from 228 open-source projects; five grade levels; developer-reported | 3.3.4 |
| OSS-Fuzz (Anthropic internal) — Reached 0.4, safeguards off | Claude Opus 4.8 | 18.1% of the time | same evaluation; developer-reported | 3.3.4 |
| OSS-Fuzz (Anthropic internal) — Reached 0.4, safeguards on | Claude Opus 4.8 | 4.9% of the time | with safeguards enabled the model did not exceed 0.4; developer-reported | 3.3.4 |
| OSS-Fuzz (Anthropic internal) — Reached 0.8, safeguards off | Claude Opus 4.7 | 3 targets | Opus 4.7 achieved 0.8 on three targets; neither Opus model reached 1.0; developer-reported | 3.3.4 |
| OSS-Fuzz (Anthropic internal) — Score 1.0, safeguards off | Claude Mythos Preview | 12 targets | evaluated without safeguards only; developer-reported | 3.3.4 |
| OSS-Fuzz (Anthropic internal) — Reached 0.2, safeguards off | Claude Mythos Preview | 45.6% of targets | same evaluation; developer-reported | 3.3.4 |
| OSS-Fuzz (Anthropic internal) — Reached 0.4, safeguards off | Claude Mythos Preview | 28.3% of targets | same evaluation; developer-reported | 3.3.4 |
| OSS-Fuzz (Anthropic internal) — Reached 0.6, safeguards off | Claude Mythos Preview | 7 targets | same evaluation; developer-reported | 3.3.4 |
| OSS-Fuzz (Anthropic internal) — Reached 0.8, safeguards off | Claude Mythos Preview | 4 targets | same evaluation; developer-reported | 3.3.4 |
| OSS-Fuzz (Anthropic internal) — No score above 0, safeguards off | Claude Mythos Preview | 23.3% of targets | same evaluation; developer-reported | 3.3.4 |
| Single-turn harmful request evaluation — Harmless response rate, API without system prompt | Claude Opus 4.8 | 97.98% | ±0.11%; 16 policy areas in 7 languages; average of runs with and without thinking; Opus 4.6 grader (changed from Sonnet 4); developer-reported | 4.1.1 |
| Single-turn harmful request evaluation — Harmless response rate, API without system prompt | Claude Opus 4.7 | 97.43% | ±0.13%; re-graded with the Opus 4.6 grader; developer-reported | 4.1.1 |
| Single-turn harmful request evaluation — Harmless response rate, API without system prompt | Claude Mythos Preview | 97.23% | ±0.14; developer-reported | 4.1.1 |
| Single-turn harmful request evaluation — Harmless response rate, API without system prompt | Claude Sonnet 4.6 | 98.18% | ±0.11%; developer-reported | 4.1.1 |
| Single-turn harmful request evaluation — Harmless response rate, claude.ai | Claude Opus 4.8 | 99.17% | ±0.07%; claude.ai default system prompt; developer-reported | 4.1.1 |
| Single-turn harmful request evaluation — Harmless response rate, claude.ai | Claude Opus 4.7 | 97.78% | ±0.12%; developer-reported | 4.1.1 |
| Single-turn harmful request evaluation — Harmless response rate, claude.ai | Claude Sonnet 4.6 | 98.68% | ±0.10%; developer-reported | 4.1.1 |
| Single-turn benign request evaluation — Over-refusal rate, API without system prompt | Claude Opus 4.8 | 0.36% | ±0.04%; same 16 policy areas and 7 languages; Opus 4.6 grader; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Over-refusal rate, API without system prompt | Claude Opus 4.7 | 0.31% | ±0.05%; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Over-refusal rate, API without system prompt | Claude Mythos Preview | 0.05% | ±0.02%; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Over-refusal rate, API without system prompt | Claude Sonnet 4.6 | 0.40% | ±0.05%; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Over-refusal rate, claude.ai | Claude Opus 4.8 | 0.49% | ±0.05%; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Over-refusal rate, claude.ai | Claude Opus 4.7 | 0.34% | ±0.05%; developer-reported | 4.1.2 |
| Single-turn benign request evaluation — Over-refusal rate, claude.ai | Claude Sonnet 4.6 | 0.91% | ±0.08%; developer-reported | 4.1.2 |
| Multi-turn testing, child safety — Appropriate response rate, claude.ai | Claude Opus 4.8 | 95% | the one claude.ai category where Opus 4.8 scored below Opus 4.7; difference not statistically significant; also in Table 4.2.B with ±4%; developer-reported | 4.1.3, 4.2 |
| Multi-turn testing, child safety — Appropriate response rate, claude.ai | Claude Opus 4.7 | 97% | also in Table 4.2.B with ±3%; developer-reported | 4.1.3, 4.2 |
| Child safety single-turn harmful requests — Harmless rate, API without system prompt | Claude Opus 4.8 | 99.90% | ±0.07%; all tested languages; average with and without thinking; developer-reported | 4.2 |
| Child safety single-turn harmful requests — Harmless rate, API without system prompt | Claude Opus 4.7 | 99.94% | ±0.05%; developer-reported | 4.2 |
| Child safety single-turn harmful requests — Harmless rate, API without system prompt | Claude Mythos Preview | 99.83% | ±0.11%; developer-reported | 4.2 |
| Child safety single-turn harmful requests — Harmless rate, API without system prompt | Claude Sonnet 4.6 | 99.96% | ±0.07%; developer-reported | 4.2 |
| Child safety single-turn harmful requests — Harmless rate, claude.ai | Claude Opus 4.8 | 99.99% | ±0.02%; developer-reported | 4.2 |
| Child safety single-turn harmful requests — Harmless rate, claude.ai | Claude Opus 4.7 | 99.93% | ±0.07%; developer-reported | 4.2 |
| Child safety single-turn harmful requests — Harmless rate, claude.ai | Claude Sonnet 4.6 | 99.92% | ±0.09%; developer-reported | 4.2 |
| Child safety single-turn benign requests — Refusal rate, API without system prompt | Claude Opus 4.8 | 0.45% | ±0.19%; developer-reported | 4.2 |
| Child safety single-turn benign requests — Refusal rate, API without system prompt | Claude Opus 4.7 | 0.00% | no margin given; developer-reported | 4.2 |
| Child safety single-turn benign requests — Refusal rate, API without system prompt | Claude Mythos Preview | 0.04% | ±0.04%; developer-reported | 4.2 |
| Child safety single-turn benign requests — Refusal rate, API without system prompt | Claude Sonnet 4.6 | 0.42% | ±0.21%; developer-reported | 4.2 |
| Child safety single-turn benign requests — Refusal rate, claude.ai | Claude Opus 4.8 | 0.27% | ±0.15%; developer-reported | 4.2 |
| Child safety single-turn benign requests — Refusal rate, claude.ai | Claude Opus 4.7 | 0.12% | ±0.10%; developer-reported | 4.2 |
| Child safety single-turn benign requests — Refusal rate, claude.ai | Claude Sonnet 4.6 | 0.51% | ±0.25%; developer-reported | 4.2 |
| Child safety multi-turn evaluations — Appropriate response rate, API without system prompt | Claude Opus 4.8 | 91% | ±4%; developer-reported | 4.2 |
| Child safety multi-turn evaluations — Appropriate response rate, API without system prompt | Claude Opus 4.7 | 83% | ±6%; developer-reported | 4.2 |
| Child safety multi-turn evaluations — Appropriate response rate, API without system prompt | Claude Mythos Preview | 88% | ±5%; developer-reported | 4.2 |
| Child safety multi-turn evaluations — Appropriate response rate, API without system prompt | Claude Sonnet 4.6 | 95% | ±3%; developer-reported | 4.2 |
| Child safety multi-turn evaluations — Appropriate response rate, claude.ai | Claude Sonnet 4.6 | 84% | ±6%; developer-reported | 4.2 |
| Suicide and self-harm single-turn, requests posing potential risk — Harmless rate, API without system prompt | Claude Opus 4.8 | 99.05% | ±0.26%; all tested languages; average with and without thinking; developer-reported | 4.3.1 |
| Suicide and self-harm single-turn, requests posing potential risk — Harmless rate, API without system prompt | Claude Opus 4.7 | 99.34% | ±0.22%; developer-reported | 4.3.1 |
| Suicide and self-harm single-turn, requests posing potential risk — Harmless rate, API without system prompt | Claude Mythos Preview | 99.78% | ±0.11%; developer-reported | 4.3.1 |
| Suicide and self-harm single-turn, requests posing potential risk — Harmless rate, API without system prompt | Claude Sonnet 4.6 | 99.67% | ±0.18%; developer-reported | 4.3.1 |
| Suicide and self-harm single-turn, requests posing potential risk — Harmless rate, claude.ai | Claude Opus 4.8 | 99.94% | ±0.07%; developer-reported | 4.3.1 |
| Suicide and self-harm single-turn, requests posing potential risk — Harmless rate, claude.ai | Claude Opus 4.7 | 99.99% | ±0.02%; developer-reported | 4.3.1 |
| Suicide and self-harm single-turn, requests posing potential risk — Harmless rate, claude.ai | Claude Sonnet 4.6 | 99.61% | ±0.20%; developer-reported | 4.3.1 |
| Suicide and self-harm single-turn benign requests — Refusal rate, API without system prompt | Claude Opus 4.8 | 0.15% | ±0.11%; developer-reported | 4.3.1 |
| Suicide and self-harm single-turn benign requests — Refusal rate, API without system prompt | Claude Opus 4.7 | 0.12% | ±0.10%; developer-reported | 4.3.1 |
| Suicide and self-harm single-turn benign requests — Refusal rate, API without system prompt | Claude Mythos Preview | 0.12% | ±0.09%; developer-reported | 4.3.1 |
| Suicide and self-harm single-turn benign requests — Refusal rate, API without system prompt | Claude Sonnet 4.6 | 0.21% | ±0.15%; developer-reported | 4.3.1 |
| Suicide and self-harm single-turn benign requests — Refusal rate, claude.ai | Claude Opus 4.8 | 0.43% | ±0.23%; developer-reported | 4.3.1 |
| Suicide and self-harm single-turn benign requests — Refusal rate, claude.ai | Claude Opus 4.7 | 0.43% | ±0.23%; developer-reported | 4.3.1 |
| Suicide and self-harm single-turn benign requests — Refusal rate, claude.ai | Claude Sonnet 4.6 | 0.02% | ±0.03%; developer-reported | 4.3.1 |
| Suicide and self-harm multi-turn evaluations — Appropriate response rate, API without system prompt | Claude Opus 4.8 | 58% | ±14%; developer-reported | 4.3.1 |
| Suicide and self-harm multi-turn evaluations — Appropriate response rate, API without system prompt | Claude Opus 4.7 | 44% | ±14%; developer-reported | 4.3.1 |
| Suicide and self-harm multi-turn evaluations — Appropriate response rate, API without system prompt | Claude Mythos Preview | 54% | ±14%; developer-reported | 4.3.1 |
| Suicide and self-harm multi-turn evaluations — Appropriate response rate, API without system prompt | Claude Sonnet 4.6 | 70% | ±13%; developer-reported | 4.3.1 |
| Suicide and self-harm multi-turn evaluations — Appropriate response rate, claude.ai | Claude Opus 4.8 | 78% | ±12%; developer-reported | 4.3.1 |
| Suicide and self-harm multi-turn evaluations — Appropriate response rate, claude.ai | Claude Opus 4.7 | 72% | ±13%; developer-reported | 4.3.1 |
| Suicide and self-harm multi-turn evaluations — Appropriate response rate, claude.ai | Claude Sonnet 4.6 | 82% | ±11%; developer-reported | 4.3.1 |
| Disordered eating single-turn, requests posing potential risk — Harmless rate, API without system prompt | Claude Opus 4.8 | 97.99% | ±0.45%; developer-reported | 4.3.2 |
| Disordered eating single-turn, requests posing potential risk — Harmless rate, API without system prompt | Claude Opus 4.7 | 97.07% | ±0.56%; developer-reported | 4.3.2 |
| Disordered eating single-turn, requests posing potential risk — Harmless rate, API without system prompt | Claude Mythos Preview | 97.14% | ±0.57%; developer-reported | 4.3.2 |
| Disordered eating single-turn, requests posing potential risk — Harmless rate, API without system prompt | Claude Sonnet 4.6 | 97.07% | ±0.57%; developer-reported | 4.3.2 |
| Disordered eating single-turn, requests posing potential risk — Harmless rate, claude.ai | Claude Opus 4.8 | 99.80% | ±0.13%; developer-reported | 4.3.2 |
| Disordered eating single-turn, requests posing potential risk — Harmless rate, claude.ai | Claude Opus 4.7 | 99.27% | ±0.28%; developer-reported | 4.3.2 |
| Disordered eating single-turn, requests posing potential risk — Harmless rate, claude.ai | Claude Sonnet 4.6 | 98.13% | ±0.45%; developer-reported | 4.3.2 |
| Disordered eating single-turn benign requests — Refusal rate, API without system prompt | Claude Opus 4.8 | 0.06% | ±0.06%; developer-reported | 4.3.2 |
| Disordered eating single-turn benign requests — Refusal rate, API without system prompt | Claude Opus 4.7 | 0.00% | no margin given; developer-reported | 4.3.2 |
| Disordered eating single-turn benign requests — Refusal rate, API without system prompt | Claude Mythos Preview | 0.00% | no margin given; developer-reported | 4.3.2 |
| Disordered eating single-turn benign requests — Refusal rate, API without system prompt | Claude Sonnet 4.6 | 0.14% | ±0.11%; developer-reported | 4.3.2 |
| Disordered eating single-turn benign requests — Refusal rate, claude.ai | Claude Opus 4.8 | 0.21% | ±0.13%; developer-reported | 4.3.2 |
| Disordered eating single-turn benign requests — Refusal rate, claude.ai | Claude Opus 4.7 | 0.29% | ±0.15%; developer-reported | 4.3.2 |
| Disordered eating single-turn benign requests — Refusal rate, claude.ai | Claude Sonnet 4.6 | 0.60% | ±0.25%; developer-reported | 4.3.2 |
| BBQ — Disambiguated accuracy | Claude Opus 4.8 | 72.1 | percent; thinking disabled, no system prompt; developer-reported | 4.4.2 |
| BBQ — Disambiguated accuracy | Claude Opus 4.7 | 81.3 | percent; thinking disabled, no system prompt; developer-reported | 4.4.2 |
| BBQ — Disambiguated accuracy | Claude Mythos Preview | 84.6 | percent; thinking disabled, no system prompt; developer-reported | 4.4.2 |
| BBQ — Disambiguated accuracy | Claude Sonnet 4.6 | 88.1 | percent; thinking disabled, no system prompt; developer-reported | 4.4.2 |
| BBQ — Ambiguous accuracy | Claude Opus 4.8 | 99.9 | percent; thinking disabled, no system prompt; developer-reported | 4.4.2 |
| BBQ — Ambiguous accuracy | Claude Opus 4.7 | 99.9 | percent; thinking disabled, no system prompt; developer-reported | 4.4.2 |
| BBQ — Ambiguous accuracy | Claude Mythos Preview | 100 | percent; thinking disabled, no system prompt; developer-reported | 4.4.2 |
| BBQ — Ambiguous accuracy | Claude Sonnet 4.6 | 97.5 | percent; thinking disabled, no system prompt; developer-reported | 4.4.2 |
| BBQ — Disambiguated bias score | Claude Opus 4.8 | -1.37 | percent; closer to zero is better; developer-reported | 4.4.2 |
| BBQ — Disambiguated bias score | Claude Opus 4.7 | -1.68 | percent; developer-reported | 4.4.2 |
| BBQ — Disambiguated bias score | Claude Mythos Preview | -1.61 | percent; developer-reported | 4.4.2 |
| BBQ — Disambiguated bias score | Claude Sonnet 4.6 | -0.67 | percent; developer-reported | 4.4.2 |
| BBQ — Ambiguous bias score | Claude Opus 4.8 | 0.07 | percent; developer-reported | 4.4.2 |
| BBQ — Ambiguous bias score | Claude Opus 4.7 | 0.04 | percent; developer-reported | 4.4.2 |
| BBQ — Ambiguous bias score | Claude Mythos Preview | 0.01 | percent; developer-reported | 4.4.2 |
| BBQ — Ambiguous bias score | Claude Sonnet 4.6 | 1.41 | percent; developer-reported | 4.4.2 |
| BBQ (share of incorrect disambiguated answers that were "cannot be determined") | Claude Opus 4.8 | roughly 97% | error analysis of the disambiguated set; developer-reported | 4.4.2 |
| Election integrity single-turn harmful requests — Harmless rate, API without system prompt | Claude Opus 4.8 | 100% | 300 violative and 300 benign prompts; thinking off; developer-reported | 4.4.3 |
| Election integrity single-turn harmful requests — Harmless rate, API without system prompt | Claude Opus 4.7 | 100% | same evaluation; developer-reported | 4.4.3 |
| Election integrity single-turn harmful requests — Harmless rate, API without system prompt | Claude Mythos Preview | 100% | same evaluation; developer-reported | 4.4.3 |
| Election integrity single-turn harmful requests — Harmless rate, API without system prompt | Claude Sonnet 4.6 | 99.89% | same evaluation; developer-reported | 4.4.3 |
| Election integrity single-turn harmful requests — Harmless rate, claude.ai | Claude Opus 4.8 | 100% | same evaluation; developer-reported | 4.4.3 |
| Election integrity single-turn harmful requests — Harmless rate, claude.ai | Claude Opus 4.7 | 100% | same evaluation; developer-reported | 4.4.3 |
| Election integrity single-turn harmful requests — Harmless rate, claude.ai | Claude Sonnet 4.6 | 100% | same evaluation; developer-reported | 4.4.3 |
| Election integrity single-turn benign requests — Refusal rate, API without system prompt | Claude Opus 4.8 | 0.33% | thinking off; developer-reported | 4.4.3 |
| Election integrity single-turn benign requests — Refusal rate, API without system prompt | Claude Opus 4.7 | 0.00% | thinking off; developer-reported | 4.4.3 |
| Election integrity single-turn benign requests — Refusal rate, API without system prompt | Claude Mythos Preview | 0.00% | thinking off; developer-reported | 4.4.3 |
| Election integrity single-turn benign requests — Refusal rate, API without system prompt | Claude Sonnet 4.6 | 0.00% | thinking off; developer-reported | 4.4.3 |
| Election integrity single-turn benign requests — Refusal rate, claude.ai | Claude Opus 4.8 | 0.00% | thinking off; developer-reported | 4.4.3 |
| Election integrity single-turn benign requests — Refusal rate, claude.ai | Claude Opus 4.7 | 0.67% | thinking off; developer-reported | 4.4.3 |
| Election integrity single-turn benign requests — Refusal rate, claude.ai | Claude Sonnet 4.6 | 0.00% | thinking off; developer-reported | 4.4.3 |
| Malicious use of Claude Code — Malicious refusal rate | Claude Opus 4.8 | 95.08% | 61 malicious prompts run 10 times each; standard Claude Code tool set and system prompt, FileRead reminder removed; developer-reported | 5.1.1 |
| Malicious use of Claude Code — Malicious refusal rate | Claude Opus 4.7 | 91.15% | same prompt set; developer-reported | 5.1.1 |
| Malicious use of Claude Code — Malicious refusal rate | Claude Mythos Preview | 95.41% | same prompt set; developer-reported | 5.1.1 |
| Malicious use of Claude Code — Malicious refusal rate | Claude Sonnet 4.6 | 89.34% | same prompt set; developer-reported | 5.1.1 |
| Malicious use of Claude Code — Dual-use and benign success rate | Claude Opus 4.8 | 92.12% | 61 dual-use/benign prompts run 10 times each; developer-reported | 5.1.1 |
| Malicious use of Claude Code — Dual-use and benign success rate | Claude Opus 4.7 | 91.83% | same prompt set; developer-reported | 5.1.1 |
| Malicious use of Claude Code — Dual-use and benign success rate | Claude Mythos Preview | 91.12% | same prompt set; developer-reported | 5.1.1 |
| Malicious use of Claude Code — Dual-use and benign success rate | Claude Sonnet 4.6 | 92.88% | same prompt set; developer-reported | 5.1.1 |
| Malicious computer use — Refusal rate, without mitigations | Claude Opus 4.8 | 81.70% | 112 tasks with thinking enabled and disabled (224 attempts); developer-reported | 5.1.2 |
| Malicious computer use — Refusal rate, without mitigations | Claude Opus 4.7 | 89.29% | same task set; developer-reported | 5.1.2 |
| Malicious computer use — Refusal rate, without mitigations | Claude Mythos Preview | 93.75% | same task set; developer-reported | 5.1.2 |
| Malicious computer use — Refusal rate, without mitigations | Claude Sonnet 4.6 | 84.82% | same task set; developer-reported | 5.1.2 |
| Malicious agentic influence campaigns — Voter suppression scenario | Claude Opus 4.8 (helpful-only) | 73.3% | task completion rate; share of 70 success criteria; 9 simulations (3 runs at 3 friction levels); helpful-only variant; developer-reported | 5.1.3 |
| Malicious agentic influence campaigns — Voter suppression scenario | Claude Opus 4.7 (helpful-only) | 57.1% | same evaluation; developer-reported | 5.1.3 |
| Malicious agentic influence campaigns — Voter suppression scenario | Claude Mythos Preview (helpful-only) | 59.5% | same evaluation; developer-reported | 5.1.3 |
| Malicious agentic influence campaigns — Voter suppression scenario | Claude Sonnet 4.6 (helpful-only) | 41.8% | same evaluation; developer-reported | 5.1.3 |
| Malicious agentic influence campaigns — Domestic polarization scenario | Claude Opus 4.8 (helpful-only) | 55.1% | task completion rate; share of 70 success criteria; 9 simulations; developer-reported | 5.1.3 |
| Malicious agentic influence campaigns — Domestic polarization scenario | Claude Opus 4.7 (helpful-only) | 46.8% | same evaluation; developer-reported | 5.1.3 |
| Malicious agentic influence campaigns — Domestic polarization scenario | Claude Mythos Preview (helpful-only) | 42.1% | same evaluation; developer-reported | 5.1.3 |
| Malicious agentic influence campaigns — Domestic polarization scenario | Claude Sonnet 4.6 (helpful-only) | 34.0% | same evaluation; developer-reported | 5.1.3 |
| Agent Red Teaming (ART) benchmark, indirect prompt injection — Attack success rate at k=100, extended thinking | Claude Opus 4.8 | 9.6% | 19 scenarios; high-transfer attack subset; evaluated by Gray Swan with the UK AI Security Institute, reported by Anthropic | 5.2.1 |
| Agent Red Teaming (ART) benchmark, indirect prompt injection — Attack success rate at k=100, extended thinking | Claude Opus 4.7 | 6.0% | evaluated by Gray Swan, reported by Anthropic | 5.2.1 |
| Agent Red Teaming (ART) benchmark, indirect prompt injection — Attack success rate at k=100, extended thinking | Claude Sonnet 4.6 | 15.9% | evaluated by Gray Swan, reported by Anthropic | 5.2.1 |
| Agent Red Teaming (ART) benchmark, indirect prompt injection — Attack success rate at k=100, no extended thinking | Claude Opus 4.8 | 14.4% | evaluated by Gray Swan, reported by Anthropic | 5.2.1 |
| Agent Red Teaming (ART) benchmark, indirect prompt injection — Attack success rate at k=100, no extended thinking | Claude Opus 4.7 | 4.8% | evaluated by Gray Swan, reported by Anthropic | 5.2.1 |
| Agent Red Teaming (ART) benchmark, indirect prompt injection — Attack success rate at k=100, no extended thinking | Claude Sonnet 4.6 | 20.7% | evaluated by Gray Swan, reported by Anthropic | 5.2.1 |
| Shade adaptive prompt injection, coding — 1 attempt, no safeguards, with thinking | Claude Opus 4.8 | 7.03% | attack success rate; Gray Swan Shade attacker trained on 40 scenarios against earlier models; not comparable to prior system cards; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, coding — 1 attempt, no safeguards, with thinking | Claude Opus 4.7 | 2.34% | same attacker and scenarios; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, coding — 1 attempt, no safeguards, with thinking | Claude Sonnet 4.6 | 12.71% | same attacker and scenarios; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, coding — 1 attempt, no safeguards, with thinking | Claude Mythos Preview | 0.0% | same attacker and scenarios; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, coding — 1 attempt, no safeguards, no thinking | Claude Opus 4.8 | 17.44% | same attacker and scenarios; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, coding — 1 attempt, no safeguards, no thinking | Claude Opus 4.7 | 10.43% | same attacker and scenarios; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, coding — 1 attempt, no safeguards, no thinking | Claude Sonnet 4.6 | 45.26% | same attacker and scenarios; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, coding — 1 attempt, no safeguards, no thinking | Claude Mythos Preview | 0.03% | same attacker and scenarios; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, coding — 200 attempts, no safeguards, with thinking | Claude Opus 4.8 | 57.5% | at least one of 200 adaptive attempts succeeded for a given goal; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, coding — 200 attempts, no safeguards, with thinking | Claude Opus 4.7 | 60.0% | same measure; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, coding — 200 attempts, no safeguards, with thinking | Claude Sonnet 4.6 | 90.0% | same measure; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, coding — 200 attempts, no safeguards, with thinking | Claude Mythos Preview | 0.0% | same measure; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, coding — 200 attempts, no safeguards, no thinking | Claude Opus 4.8 | 95.0% | same measure; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, coding — 200 attempts, no safeguards, no thinking | Claude Opus 4.7 | 92.5% | same measure; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, coding — 200 attempts, no safeguards, no thinking | Claude Sonnet 4.6 | 100% | same measure; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, coding — 200 attempts, no safeguards, no thinking | Claude Mythos Preview | 2.5% | same measure; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, coding — 1 attempt, with safeguards, with thinking | Claude Opus 4.8 | 2.09% | prompt-injection safeguards enabled; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, coding — 1 attempt, with safeguards, with thinking | Claude Opus 4.7 | 0.43% | safeguards enabled; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, coding — 1 attempt, with safeguards, with thinking | Claude Sonnet 4.6 | 2.99% | safeguards enabled; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, coding — 1 attempt, with safeguards, with thinking | Claude Mythos Preview | 0.0% | safeguards enabled; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, coding — 1 attempt, with safeguards, no thinking | Claude Opus 4.8 | 4.11% | safeguards enabled; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, coding — 1 attempt, with safeguards, no thinking | Claude Opus 4.7 | 1.76% | safeguards enabled; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, coding — 1 attempt, with safeguards, no thinking | Claude Sonnet 4.6 | 8.70% | safeguards enabled; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, coding — 1 attempt, with safeguards, no thinking | Claude Mythos Preview | 0.0% | safeguards enabled; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, coding — 200 attempts, with safeguards, with thinking | Claude Opus 4.8 | 37.5% | safeguards enabled; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, coding — 200 attempts, with safeguards, with thinking | Claude Opus 4.7 | 25.0% | safeguards enabled; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, coding — 200 attempts, with safeguards, with thinking | Claude Sonnet 4.6 | 80.0% | safeguards enabled; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, coding — 200 attempts, with safeguards, with thinking | Claude Mythos Preview | 0.0% | safeguards enabled; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, coding — 200 attempts, with safeguards, no thinking | Claude Opus 4.8 | 65.0% | safeguards enabled; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, coding — 200 attempts, with safeguards, no thinking | Claude Opus 4.7 | 52.5% | safeguards enabled; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, coding — 200 attempts, with safeguards, no thinking | Claude Sonnet 4.6 | 100% | safeguards enabled; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, coding — 200 attempts, with safeguards, no thinking | Claude Mythos Preview | 0.0% | safeguards enabled; developer-reported | 5.2.2.2 |
| Shade adaptive prompt injection, computer use — 1 attempt, no safeguards, with thinking | Claude Opus 4.8 | 7.14% | attack success rate; 14 test cases; same attacker as the Opus 4.7 system card; developer-reported | 5.2.2.3 |
| Shade adaptive prompt injection, computer use — 1 attempt, no safeguards, with thinking | Claude Opus 4.7 | 0.46% | same attacker and test cases; developer-reported | 5.2.2.3 |
| Shade adaptive prompt injection, computer use — 1 attempt, no safeguards, with thinking | Claude Sonnet 4.6 | 12.0% | same attacker and test cases; developer-reported | 5.2.2.3 |
| Shade adaptive prompt injection, computer use — 1 attempt, no safeguards, with thinking | Claude Mythos Preview | 0.43% | same attacker and test cases; developer-reported | 5.2.2.3 |
| Shade adaptive prompt injection, computer use — 1 attempt, no safeguards, no thinking | Claude Opus 4.8 | 6.21% | same attacker and test cases; developer-reported | 5.2.2.3 |
| Shade adaptive prompt injection, computer use — 1 attempt, no safeguards, no thinking | Claude Opus 4.7 | 0.39% | same attacker and test cases; developer-reported | 5.2.2.3 |
| Shade adaptive prompt injection, computer use — 1 attempt, no safeguards, no thinking | Claude Sonnet 4.6 | 14.4% | same attacker and test cases; developer-reported | 5.2.2.3 |
| Shade adaptive prompt injection, computer use — 1 attempt, no safeguards, no thinking | Claude Mythos Preview | 0.39% | same attacker and test cases; developer-reported | 5.2.2.3 |
| Shade adaptive prompt injection, computer use — 200 attempts, no safeguards, with thinking | Claude Opus 4.8 | 50.0% | at least one of 200 attempts succeeded; developer-reported | 5.2.2.3 |
| Shade adaptive prompt injection, computer use — 200 attempts, no safeguards, with thinking | Claude Opus 4.7 | 7.14% | same measure; developer-reported | 5.2.2.3 |
| Shade adaptive prompt injection, computer use — 200 attempts, no safeguards, with thinking | Claude Sonnet 4.6 | 42.9% | same measure; developer-reported | 5.2.2.3 |
| Shade adaptive prompt injection, computer use — 200 attempts, no safeguards, with thinking | Claude Mythos Preview | 21.43% | same measure; developer-reported | 5.2.2.3 |
| Shade adaptive prompt injection, computer use — 200 attempts, no safeguards, no thinking | Claude Opus 4.8 | 64.3% | same measure; developer-reported | 5.2.2.3 |
| Shade adaptive prompt injection, computer use — 200 attempts, no safeguards, no thinking | Claude Opus 4.7 | 21.43% | same measure; developer-reported | 5.2.2.3 |
| Shade adaptive prompt injection, computer use — 200 attempts, no safeguards, no thinking | Claude Sonnet 4.6 | 64.3% | same measure; developer-reported | 5.2.2.3 |
| Shade adaptive prompt injection, computer use — 200 attempts, no safeguards, no thinking | Claude Mythos Preview | 14.29% | same measure; developer-reported | 5.2.2.3 |
| Shade adaptive prompt injection, computer use — 1 attempt, with safeguards, with thinking | Claude Opus 4.8 | 5.11% | safeguards enabled; developer-reported | 5.2.2.3 |
| Shade adaptive prompt injection, computer use — 1 attempt, with safeguards, with thinking | Claude Opus 4.7 | 0.61% | safeguards enabled; developer-reported | 5.2.2.3 |
| Shade adaptive prompt injection, computer use — 1 attempt, with safeguards, with thinking | Claude Sonnet 4.6 | 6.21% | safeguards enabled; developer-reported | 5.2.2.3 |
| Shade adaptive prompt injection, computer use — 1 attempt, with safeguards, with thinking | Claude Mythos Preview | 0.32% | safeguards enabled; developer-reported | 5.2.2.3 |
| Shade adaptive prompt injection, computer use — 1 attempt, with safeguards, no thinking | Claude Opus 4.8 | 3.75% | safeguards enabled; developer-reported | 5.2.2.3 |
| Shade adaptive prompt injection, computer use — 1 attempt, with safeguards, no thinking | Claude Opus 4.7 | 0.50% | safeguards enabled; developer-reported | 5.2.2.3 |
| Shade adaptive prompt injection, computer use — 1 attempt, with safeguards, no thinking | Claude Sonnet 4.6 | 6.32% | safeguards enabled; developer-reported | 5.2.2.3 |
| Shade adaptive prompt injection, computer use — 1 attempt, with safeguards, no thinking | Claude Mythos Preview | 0.36% | safeguards enabled; developer-reported | 5.2.2.3 |
| Shade adaptive prompt injection, computer use — 200 attempts, with safeguards, with thinking | Claude Opus 4.8 | 57.1% | safeguards enabled; Anthropic attributes the rise over the no-safeguards figure to one test case going from 0/200 to 1/200; developer-reported | 5.2.2.3 |
| Shade adaptive prompt injection, computer use — 200 attempts, with safeguards, with thinking | Claude Opus 4.7 | 14.29% | safeguards enabled; developer-reported | 5.2.2.3 |
| Shade adaptive prompt injection, computer use — 200 attempts, with safeguards, with thinking | Claude Sonnet 4.6 | 64.29% | safeguards enabled; developer-reported | 5.2.2.3 |
| Shade adaptive prompt injection, computer use — 200 attempts, with safeguards, with thinking | Claude Mythos Preview | 21.43% | safeguards enabled; developer-reported | 5.2.2.3 |
| Shade adaptive prompt injection, computer use — 200 attempts, with safeguards, no thinking | Claude Opus 4.8 | 64.3% | safeguards enabled; developer-reported | 5.2.2.3 |
| Shade adaptive prompt injection, computer use — 200 attempts, with safeguards, no thinking | Claude Opus 4.7 | 35.71% | safeguards enabled; developer-reported | 5.2.2.3 |
| Shade adaptive prompt injection, computer use — 200 attempts, with safeguards, no thinking | Claude Sonnet 4.6 | 78.57% | safeguards enabled; developer-reported | 5.2.2.3 |
| Shade adaptive prompt injection, computer use — 200 attempts, with safeguards, no thinking | Claude Mythos Preview | 14.29% | safeguards enabled; developer-reported | 5.2.2.3 |
| Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, no safeguards, with thinking | Claude Opus 4.8 | 62.8% | 129 held-out environments, 10 attempts each; attacks adaptively sourced against Opus 4.7 by professional red-teamers; developer-reported | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, no safeguards, with thinking | Claude Opus 4.7 | 78.3% | same environments; developer-reported | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, no safeguards, with thinking | Claude Sonnet 4.6 | 76.0% | same environments; developer-reported | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, no safeguards, with thinking | Claude Mythos Preview | 14.0% | same environments; developer-reported | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, no safeguards, no thinking | Claude Opus 4.8 | 46.5% | same environments; developer-reported | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, no safeguards, no thinking | Claude Opus 4.7 | 74.2% | same environments; developer-reported | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, no safeguards, no thinking | Claude Sonnet 4.6 | 76.7% | same environments; developer-reported | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, no safeguards, no thinking | Claude Mythos Preview | 10.8% | same environments; developer-reported | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, no safeguards, with thinking | Claude Opus 4.8 | 31.5% | per-attempt attack success rate across 129 environments; developer-reported | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, no safeguards, with thinking | Claude Opus 4.7 | 40.0% | same measure; developer-reported | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, no safeguards, with thinking | Claude Sonnet 4.6 | 50.7% | same measure; developer-reported | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, no safeguards, with thinking | Claude Mythos Preview | 5.9% | same measure; developer-reported | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, no safeguards, no thinking | Claude Opus 4.8 | 17.8% | same measure; developer-reported | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, no safeguards, no thinking | Claude Opus 4.7 | 41.1% | same measure; developer-reported | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, no safeguards, no thinking | Claude Sonnet 4.6 | 47.3% | same measure; developer-reported | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, no safeguards, no thinking | Claude Mythos Preview | 3.7% | same measure; developer-reported | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, with safeguards, with thinking | Claude Opus 4.8 | 3.9% | deployed safeguards enabled; developer-reported | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, with safeguards, with thinking | Claude Opus 4.7 | 12.4% | safeguards enabled; developer-reported | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, with safeguards, with thinking | Claude Sonnet 4.6 | 46.5% | safeguards enabled; developer-reported | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, with safeguards, with thinking | Claude Mythos Preview | 6.2% | safeguards enabled; developer-reported | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, with safeguards, no thinking | Claude Opus 4.8 | 0.0% | safeguards enabled; no attacks succeeded across the 129 environments; developer-reported | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, with safeguards, no thinking | Claude Opus 4.7 | 8.5% | safeguards enabled; developer-reported | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, with safeguards, no thinking | Claude Sonnet 4.6 | 30.2% | safeguards enabled; developer-reported | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, with safeguards, no thinking | Claude Mythos Preview | 3.1% | safeguards enabled; developer-reported | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, with safeguards, with thinking | Claude Opus 4.8 | 0.5% | safeguards enabled; developer-reported | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, with safeguards, with thinking | Claude Opus 4.7 | 4.2% | safeguards enabled; developer-reported | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, with safeguards, with thinking | Claude Sonnet 4.6 | 23.6% | safeguards enabled; developer-reported | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, with safeguards, with thinking | Claude Mythos Preview | 2.4% | safeguards enabled; developer-reported | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, with safeguards, no thinking | Claude Opus 4.8 | 0.0% | safeguards enabled; developer-reported | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, with safeguards, no thinking | Claude Opus 4.7 | 1.4% | safeguards enabled; developer-reported | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, with safeguards, no thinking | Claude Sonnet 4.6 | 12.5% | safeguards enabled; developer-reported | 5.2.2.4 |
| Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, with safeguards, no thinking | Claude Mythos Preview | 2.5% | safeguards enabled; developer-reported | 5.2.2.4 |
| Constitutional adherence (15 dimensions) — Dimensions where Opus 4.8 is the strongest model measured | Claude Opus 4.8 | 8 / 15 | roughly 1,000 transcripts per model across 40 constitutional areas; scored -3 to +3 by Claude Opus 4.7; on all 15 dimensions Opus 4.8 was best or statistically equivalent to the best; developer-reported | 6.2.3.1.6, 6.3.2.3 |
| UK AISI AI-safety-sabotage evaluation — Partial refusals on benign safety research tasks | Claude Opus 4.8 | 14.1% | pre-release snapshot tested by the UK AI Security Institute; full refusals near-zero | 6.2.4 |
| UK AISI AI-safety-sabotage evaluation — Partial refusals on benign safety research tasks | Claude Opus 4.7 | 13.7% | measured by the UK AI Security Institute | 6.2.4 |
| UK AISI AI-safety-sabotage evaluation — Partial refusals on benign safety research tasks | Claude Mythos Preview | 33.3% | measured by the UK AI Security Institute | 6.2.4 |
| UK AISI AI-safety-sabotage evaluation — Continuation setting, actively continued the attempt | Claude Opus 4.8 | 1% | synthetically inserted prior reasoning begins an attempt to compromise AI safety research; measured by the UK AI Security Institute | 6.2.4 |
| UK AISI AI-safety-sabotage evaluation — Continuation setting, actively continued the attempt | Claude Opus 4.7 | 0% | measured by the UK AI Security Institute | 6.2.4 |
| UK AISI AI-safety-sabotage evaluation — Continuation setting, actively continued the attempt | Claude Mythos Preview | 7% | measured by the UK AI Security Institute | 6.2.4 |
| UK AISI evaluation-awareness discrimination — Real vs. synthetic reproductions, prompted | Claude Opus 4.8 | 79% | accuracy distinguishing real deployment data from synthetic reproductions of the same tasks; measured by the UK AI Security Institute | 6.2.3.2, 6.2.4 |
| UK AISI evaluation-awareness discrimination — Real vs. synthetic reproductions, prompted | Claude Mythos Preview | 79% | measured by the UK AI Security Institute | 6.2.4 |
| UK AISI evaluation-awareness discrimination — Real vs. synthetic reproductions, prompted | Claude Opus 4.7 | 68% | measured by the UK AI Security Institute | 6.2.4 |
| UK AISI unprompted verbalised evaluation awareness — Unprompted sabotage evaluation | Claude Opus 4.8 | 26.3% | measured by the UK AI Security Institute | 6.2.4 |
| UK AISI unprompted verbalised evaluation awareness — Unprompted sabotage evaluation | Claude Opus 4.7 | 21.7% | measured by the UK AI Security Institute | 6.2.4 |
| UK AISI unprompted verbalised evaluation awareness — Continuation evaluation | Claude Opus 4.8 | 1.2% | measured by the UK AI Security Institute | 6.2.4 |
| UK AISI unprompted verbalised evaluation awareness — Continuation evaluation | Claude Opus 4.7 | 3.4% | measured by the UK AI Security Institute | 6.2.4 |
| Lying about being human — Rate of acknowledging it is an AI | Claude Opus 4.8 | 97% | simulated high-stakes professional scenarios where a system prompt assigns a human persona; set described as mostly saturated; developer-reported | 6.3.3.5 |
| Lying about being human — Rate of acknowledging it is an AI | Claude Mythos Preview | 100% | same evaluation; developer-reported | 6.3.3.5 |
| Code summary honesty — Failure to raise important events | Claude Opus 4.8 | 3.7% | share of prefilled unsuccessful agentic coding transcripts the model summarised without flagging failures; developer-reported | 6.3.6.2 |
| Code summary honesty — Failure to raise important events | Claude Mythos Preview | 27.6% | same evaluation; developer-reported | 6.3.6.2 |
| Lazy investigation — Incorrect answer rate | Claude Opus 4.7 | 25% | next-best model after Opus 4.8, which the card reports as the first Claude model with a perfect score; developer-reported | 6.3.6.3 |
| Sandbagging screen of dangerous-capability transcripts — Transcripts flagged for manual review | Claude Opus 4.8 | 5 of 780 | 780 transcripts (240 AI R&D, 500 helpful-only CB-1, 40 AAV capsid CB-2) graded 0–10 by Claude Mythos Preview; all five were on the synthesis screening evasion task; none showed clear signs of deliberate sandbagging; developer-reported | 6.4.1 |
| SHADE-Arena — Stealth rate, Extended Thinking | Claude Opus 4.8 | just over 1% | 24 tasks, 75 attempts per task; monitor sees the agent's reasoning; roughly flat across reasoning-effort levels; developer-reported | 6.4.2.1 |
| SHADE-Arena — Stealth rate, Hidden Scratchpad | Claude Opus 4.8 | 12% | monitor sees only user-visible outputs; upper bound; developer-reported | 6.4.2.1 |
| SHADE-Arena — Side task referenced in reasoning, max effort | Claude Opus 4.8 | roughly 93% | keyword-based review of trajectories; developer-reported | 6.4.2.1 |
| SHADE-Arena — Side task referenced in reasoning, max effort | Claude Opus 4.6 | roughly 99% | keyword-based review; developer-reported | 6.4.2.1 |
| SHADE-Arena — Side task referenced in reasoning, max effort | Claude Sonnet 4.6 | roughly 99% | keyword-based review; developer-reported | 6.4.2.1 |
| SHADE-Arena — Side task referenced in reasoning, max effort | Claude Opus 4.7 | roughly 77% | keyword-based review; developer-reported | 6.4.2.1 |
| SHADE-Arena — Side task referenced in reasoning, max effort | Claude Mythos Preview | roughly 21% | keyword-based review; developer-reported | 6.4.2.1 |
| SHADE-Arena — Side task attempted, max effort | Claude Opus 4.8 | roughly 33% | share of trajectories; developer-reported | 6.4.2.1 |
| SHADE-Arena — Side task attempted, max effort | Claude Opus 4.6 | roughly 90% | share of trajectories; developer-reported | 6.4.2.1 |
| SHADE-Arena — Side task attempted, max effort | Claude Sonnet 4.6 | roughly 90% | share of trajectories; developer-reported | 6.4.2.1 |
| Minimal-LinuxBench — Stealth success, monitor without CoT | Claude Opus 4.8 | roughly 5% | ~170 task-matched tasks; adapted from Redwood Research's LinuxBench; developer-reported | 6.4.2.2 |
| Minimal-LinuxBench — Stealth success, monitor without CoT | Claude Opus 4.7 | roughly 2% | same setup; developer-reported | 6.4.2.2 |
| SHADE-Arena after fine-tuning elicitation — Stealth rate, no extended thinking | Claude Opus 4.8 (baseline) | 4.0% | task-weighted; chain-of-thought distillation on Minimal-LinuxBench stealth-successful trajectories; developer-reported | 6.4.2.3 |
| SHADE-Arena after fine-tuning elicitation — Stealth rate, no extended thinking | Claude Opus 4.8 (fine-tuned) | 4.7% | increase not statistically significant under task-level resampling; developer-reported | 6.4.2.3 |
| SHADE-Arena after fine-tuning elicitation — Stealth rate, extended thinking | Claude Opus 4.8 (baseline) | 1.3% | task-weighted; developer-reported | 6.4.2.3 |
| SHADE-Arena after fine-tuning elicitation — Stealth rate, extended thinking | Claude Opus 4.8 (fine-tuned) | 2.5% | increase not statistically significant; developer-reported | 6.4.2.3 |
| SHADE-Arena after fine-tuning elicitation — Side-task completion, no extended thinking | Claude Opus 4.8 (baseline) | 15% | regardless of detection; developer-reported | 6.4.2.3 |
| SHADE-Arena after fine-tuning elicitation — Side-task completion, no extended thinking | Claude Opus 4.8 (fine-tuned) | 41% | regardless of detection; developer-reported | 6.4.2.3 |
| SHADE-Arena after fine-tuning elicitation — Side-task completion, extended thinking | Claude Opus 4.8 (baseline) | 20% | regardless of detection; developer-reported | 6.4.2.3 |
| SHADE-Arena after fine-tuning elicitation — Side-task completion, extended thinking | Claude Opus 4.8 (fine-tuned) | 52% | regardless of detection; developer-reported | 6.4.2.3 |
| Rare-action evaluation (Serrano et al. 2026 replication) — Baseline | Claude Opus 4.8 | 0 | 4,000 samples per setting; model instructed to take the action 1% of the time; developer-reported | 6.4.2.4 |
| Rare-action evaluation (Serrano et al. 2026 replication) — UUID | Claude Opus 4.8 | 11 (0.28%) | 4,000 samples; developer-reported | 6.4.2.4 |
| Rare-action evaluation (Serrano et al. 2026 replication) — SSoT | Claude Opus 4.8 | 18 (0.45%) | 4,000 samples; developer-reported | 6.4.2.4 |
| Rare-action evaluation (Serrano et al. 2026 replication) — Strategy hint | Claude Opus 4.8 | 0 | 4,000 samples; developer-reported | 6.4.2.4 |
| Rare-action evaluation (Serrano et al. 2026 replication) — Organic | Claude Opus 4.8 | 0 | 4,000 samples; developer-reported | 6.4.2.4 |
| Rare-action evaluation (Serrano et al. 2026 replication) — UUID | Claude Opus 4.5 | 19 (0.48%) | 4,000 samples; developer-reported | 6.4.2.4 |
| Rare-action evaluation (Serrano et al. 2026 replication) — SSoT | Claude Opus 4.5 | 14 (0.35%) | 4,000 samples; developer-reported | 6.4.2.4 |
| Rare-action evaluation (Serrano et al. 2026 replication) — UUID | Claude Opus 4.6 | 8 (0.20%) | 4,000 samples; developer-reported | 6.4.2.4 |
| Rare-action evaluation (Serrano et al. 2026 replication) — SSoT | Claude Opus 4.6 | 22 (0.55%) | 4,000 samples; developer-reported | 6.4.2.4 |
| Rare-action evaluation (Serrano et al. 2026 replication) — UUID | Claude Mythos Preview | 2 (0.05%) | 4,000 samples; developer-reported | 6.4.2.4 |
| Rare-action evaluation (Serrano et al. 2026 replication) — SSoT | Claude Mythos Preview | 18 (0.45%) | 4,000 samples; developer-reported | 6.4.2.4 |
| Rare-action evaluation (Serrano et al. 2026 replication) — UUID | Claude Opus 4.7 | 4 (0.1%) | 4,000 samples; developer-reported | 6.4.2.4 |
| Rare-action evaluation (Serrano et al. 2026 replication) — SSoT | Claude Opus 4.7 | 16 (0.4%) | 4,000 samples; developer-reported | 6.4.2.4 |
| Evaluation-awareness probes vs. real Claude Code sessions — Real-session-derived audit transcripts | Claude Opus 4.8 | about 30% | share of audit transcripts producing probe scores equal to or lower than the median real internal Claude Code session; exact figures not comparable across cards; developer-reported | 6.6.2 |
| Unverbalized grader awareness (NLA pipeline) — Unprompted and unverbalized | Claude Opus 4.8 | around 5% of sampled episodes | tens of thousands of RL episodes, 50 sampled tokens each; averaged across post-training transcripts; Anthropic calls the pipeline preliminary; developer-reported | 6.6.3 |
| Unverbalized grader awareness (NLA pipeline) — Exploitative | Claude Opus 4.8 | 0.5% of sampled episodes | includes episodes whose prompts referenced grading; developer-reported | 6.6.3 |
Relationships
- supports: Claude Opus 4.8 — the safety documentation for the release
- related: Responsible Scaling Policy (RSP), Anthropic Responsible Scaling Policy v3.4 (July 2026) — the framework under which the determinations are made
- related: Claude Mythos Preview, Claude Opus 4.7, CBRN Uplift, AI and Cybersecurity, Anthropic