AI Policy Wiki
Dashboard
⚲

System Card: Claude Opus 4.8 (Anthropic, May 2026)

high confidence · updated 2026-07-26

Anthropic's system card for Claude Opus 4.8. Assessed as moderately more capable than Opus 4.7 on autonomy-relevant evaluations but less capable than Claude Mythos Preview, with alignment risk 'very low, but higher than for models prior to Claude Mythos Preview.' Records CB-1 capability with ASL-3-equivalent mitigations, an AECI point estimate of 155.5, and five worked examples of the model's shortcomings relative to human researchers, including fabrication and ignored corrections.

Published May 28, 2026, with a June 17 changelog correcting a long-form virology result affected by a tool-use error (0.89 → 0.90, stated not to affect the CB-1 analysis). See Claude Opus 4.8.

The card is structured around RSP risk assessment rather than around capabilities, reflecting the v3.x shift toward Risk Reports.

Autonomy

Threat model 1 — early-stage misalignment risk concerns "AI systems that are highly relied on and have extensive access to sensitive assets as well as moderate capacity for autonomous, goal-directed operation and subterfuge — such that it is plausible these AI systems could (if directed toward this goal, either deliberately or inadvertently) carry out actions leading to irreversibly and substantially higher odds of a later global catastrophe."

The determination places Opus 4.8 by comparison: "moderately more capable than Claude Opus 4.7 on autonomy-relevant evaluations but remains less capable than Claude Mythos Preview," with "broadly unconcerning alignment properties similar to those of Claude Opus 4.7." Anthropic therefore does "not believe Claude Opus 4.8 raises the level of risk under this threat model beyond what was assessed in the Claude Mythos Preview Alignment Risk Update."

AI R&D and the AECI. Capability trajectory is tracked with the Anthropic ECI, "a fork of Epoch AI's Epoch Capability Index." Opus 4.8's AECI is 155.5, between Opus 4.7 (154.1) and Claude Mythos Preview (158.3) on the same evaluation set — with the transparency note that this point estimate used "a smaller set of evaluations (n=11) than for previous launches (n=25)." The conclusion: Opus 4.8 "does not advance the frontier and does not change the slope-ratio analysis."

The RSP threshold quoted is met if "(1) our models would be able to fully substitute for our entire set of Research Scientists and Research Engineers, at competitive costs (i.e., within a factor of 5); or (2) there is 'dramatic acceleration' of the pace of AI progress for reasons that likely relate to the automation of AI R&D."

Documented shortcomings. Unusually, the card devotes a section to five worked examples of where the model falls short of human researchers, labelled by failure type: fabrication, ignored correction, cheap verification skipped, and instruction-following failure — three of the five involving fabrication or ignored corrections. Publishing specific failure transcripts as part of the capability argument is a form of evidence rarely included in system cards.

Chemical and biological

CB-1 is defined as the ability "to significantly help individuals or groups with basic technical backgrounds (e.g., undergraduate STEM degrees) create/obtain and deploy chemical and/or biological weapons with serious potential for catastrophic damages."

Anthropic's assessment is affirmative rather than negative: the capability assessments "are consistent with the model being capable of providing specific, actionable information relevant to the threat model, such that it may save even experts in these domains substantial time." The response is mitigation rather than restriction of release — "strong real-time classifier guards… and access controls for classifier guard exemptions," a bug bounty program and threat intelligence for continual assessment of guard effectiveness, "a variety of rapid response options for jailbreaks," and security controls against weight theft. These are stated to be "equal to or stronger than our historical ASL-3" protections.

CB-2 evaluations cover black-box RNA sequence modeling and design, and AAV capsid packaging prediction. See CBRN Uplift, AI Biosecurity.

Alignment risk

"We currently believe that the risk of significantly harmful outcomes that are substantially enabled by misaligned actions taken by our models is very low, but higher than for models prior to Claude Mythos Preview."

Two newly enumerated risk pathways appear: Pathway 7, undermining R&D within other high-resource AI developers, and Pathway 8, undermining decisions within major governments — both concerning influence on external institutions rather than on Anthropic's own systems.

Cyber

Without safeguards, Opus 4.8 "generally demonstrated slightly stronger cyber capabilities than Opus 4.7, although the models performed similarly in some evaluations. With safeguards applied, the models were generally comparable." Against Mythos Preview it was "generally much less capable… in line with our expectations given the step-change in cyber capabilities that Mythos Preview represented." Evaluations reported include ExploitBench, CyberGym, Firefox exploits, and OSS-Fuzz; Cybench was removed based on prior findings.

Reported results

Evaluation results the document states for which there is no separate benchmark page; results on benchmarks that have a page are recorded there and cite this page. Scores and settings are as printed in the document.

EvaluationModelScoreSettingSection
ChartQAPro — No toolsClaude Opus 4.869.4%full test set (1,948 questions); adaptive thinking, max effort; average over 5 runs; chain-of-thought prompting and rule-based grading per VLMEvalKit; developer-reported8.1, 8.12.1
ChartQAPro — No toolsClaude Opus 4.767.6%same settings; developer-reported8.1, 8.12.1
ChartQAPro — With toolsClaude Opus 4.872.3%Python tools; adaptive thinking, max effort; average over 5 runs; developer-reported8.1, 8.12.1
ChartQAPro — With toolsClaude Opus 4.769.8%same settings; developer-reported8.1, 8.12.1
AutomationBenchClaude Opus 4.815.5%Zapier leaderboard, private held-out set; max effort; developer-reported8.1, 8.13.8
AutomationBenchClaude Opus 4.79.9%Zapier leaderboard, private held-out set; max effort; developer-reported8.1, 8.13.8
AutomationBenchGPT-5.512.9%Zapier leaderboard; setting not stated; reported by Anthropic in the Claude Opus 4.8 system card8.1
AutomationBenchGemini 3.1 Pro9.6%Zapier leaderboard; setting not stated; reported by Anthropic in the Claude Opus 4.8 system card8.1
AutomationBenchGemini 3.5 Flash14.5%Zapier leaderboard; setting not stated; reported by Anthropic in the Claude Opus 4.8 system card8.1
GraphWalks BFS — 256K subsetClaude Opus 4.885.9F1; average over 5 trials, default sampling; Anthropic's corrected F1 scoring and clarified BFS prompt; developer-reported8.1, 8.9
GraphWalks BFS — 256K subsetClaude Opus 4.776.9F1; average over 5 trials; developer-reported8.1, 8.9
GraphWalks BFS — 256K subsetClaude Opus 4.661.1F1; average over 5 trials; developer-reported8.9
GraphWalks BFS — 256K subsetGPT-5.573.7F1; xhigh thinking, as reported in OpenAI's GPT-5.5 announcement; reported by Anthropic in the Claude Opus 4.8 system card8.1, 8.9
GraphWalks BFS — 1M subsetClaude Opus 4.868.1F1; average over 5 trials; not reproducible via the public API (problems exceed its 1M token limit); developer-reported8.9
GraphWalks BFS — 1M subsetClaude Opus 4.740.3F1; average over 5 trials; developer-reported8.9
GraphWalks BFS — 1M subsetClaude Opus 4.616.3F1; average over 5 trials; developer-reported8.9
GraphWalks BFS — 1M subsetGPT-5.545.4F1; xhigh thinking, as reported in OpenAI's GPT-5.5 announcement; reported by Anthropic in the Claude Opus 4.8 system card8.9
GraphWalks Parents — 256K subsetClaude Opus 4.899.3F1; average over 5 trials, default sampling; developer-reported8.1, 8.9
GraphWalks Parents — 256K subsetClaude Opus 4.793.6F1; average over 5 trials; developer-reported8.1, 8.9
GraphWalks Parents — 256K subsetClaude Opus 4.695.4F1; average over 5 trials; developer-reported8.9
GraphWalks Parents — 256K subsetGPT-5.590.1F1; xhigh thinking, as reported in OpenAI's GPT-5.5 announcement; reported by Anthropic in the Claude Opus 4.8 system card8.1, 8.9
GraphWalks Parents — 1M subsetClaude Opus 4.883.3F1; average over 5 trials; not reproducible via the public API; developer-reported8.9
GraphWalks Parents — 1M subsetClaude Opus 4.756.6F1; average over 5 trials; developer-reported8.9
GraphWalks Parents — 1M subsetClaude Opus 4.648.6F1; average over 5 trials; developer-reported8.9
GraphWalks Parents — 1M subsetGPT-5.558.5F1; xhigh thinking, as reported in OpenAI's GPT-5.5 announcement; reported by Anthropic in the Claude Opus 4.8 system card8.9
FrontierSWE — mean@5 average rankClaude Opus 4.82.74average rank (rank #1) on the FrontierSWE leaderboard; 17 tasks, 20 hours per task, five trials, xhigh reasoning effort; leaderboard run by Proximal, reported by Anthropic8.4
FrontierSWE — best@5 average rankClaude Opus 4.82.26average rank (rank #1) on the FrontierSWE leaderboard; xhigh reasoning effort; reported by Anthropic8.4
FrontierSWE — mean@5 average rankGPT-5.53.06average rank (rank #2); xhigh reasoning effort; FrontierSWE leaderboard; reported by Anthropic in the Claude Opus 4.8 system card8.4
FrontierSWE — best@5 average rankGPT-5.53.41average rank (rank #2); xhigh reasoning effort; FrontierSWE leaderboard; reported by Anthropic in the Claude Opus 4.8 system card8.4
FrontierSWE — mean@5 average rankClaude Opus 4.74.15average rank (rank #3); xhigh reasoning effort; FrontierSWE leaderboard, reported by Anthropic8.4
FrontierSWE — best@5 average rankClaude Opus 4.73.68average rank (rank #3); xhigh reasoning effort; FrontierSWE leaderboard, reported by Anthropic8.4
ProgramBenchClaude Opus 4.879–88%hidden test pass rate, range across 1–5 episodes each with a context budget of up to 1M tokens; 166 of 200 tasks (34 excluded for reference-binary flakiness); measured on a near-final snapshot; developer-reported8.5
ProgramBenchClaude Opus 4.771–84%hidden test pass rate, range across 1–5 episodes of up to 1M tokens; 166 tasks; developer-reported8.5
DeepSearchQA (F1)Claude Mythos Preview94.4%±1.3%; web search, web fetch, programmatic tool calling; max effort, adaptive thinking; 1M token budget, no context compaction; developer-reported8.10.3
DeepSearchQA (F1)Claude Opus 4.688.7%±1.8%; same settings; developer-reported8.10.3
DeepSearchQA (F1)Claude Opus 4.789.4%±1.8%; same settings; developer-reported8.10.3
DeepSearchQA (F1)Claude Opus 4.893.1%±1.4%; 900 prompts; web search, web fetch, programmatic tool calling; max effort, adaptive thinking; 1M token budget, no context compaction; developer-reported8.10.3
DeepSearchQA (fully correct)Claude Mythos Preview86.9%±2.2%; same run as F1; developer-reported8.10.3
DeepSearchQA (fully correct)Claude Opus 4.677.3%±2.7%; same run as F1; developer-reported8.10.3
DeepSearchQA (fully correct)Claude Opus 4.779.8%±2.6%; same run as F1; developer-reported8.10.3
DeepSearchQA (fully correct)Claude Opus 4.884.8%±2.4%; same run as F1; developer-reported8.10.3
DeepSearchQA (fully incorrect)Claude Mythos Preview3.1%±1.1%; same run as F1; developer-reported8.10.3
DeepSearchQA (fully incorrect)Claude Opus 4.66.8%±1.6%; same run as F1; developer-reported8.10.3
DeepSearchQA (fully incorrect)Claude Opus 4.76.6%±1.6%; same run as F1; developer-reported8.10.3
DeepSearchQA (fully incorrect)Claude Opus 4.83.9%±1.3%; same run as F1; developer-reported8.10.3
DeepSearchQA (correct with excessive answers)Claude Mythos Preview4.7%±1.4%; same run as F1; developer-reported8.10.3
DeepSearchQA (correct with excessive answers)Claude Opus 4.65.7%±1.5%; same run as F1; developer-reported8.10.3
DeepSearchQA (correct with excessive answers)Claude Opus 4.73.6%±1.2%; same run as F1; developer-reported8.10.3
DeepSearchQA (correct with excessive answers)Claude Opus 4.84.3%±1.3%; same run as F1; developer-reported8.10.3
DRACOClaude Opus 4.880.4%normalized score; 100 tasks; web search, web fetch, programmatic tool calling, code execution; adaptive thinking at max effort, 1M token limit, compaction at 200k tokens; Claude Opus 4.6 judge, mean of 5 grading runs, final report graded only; not comparable to the DRACO paper's figures; developer-reported8.10.4
ChartMuseum — No toolsClaude Opus 4.875.8%test split (1,162 questions); adaptive thinking, max effort; average over 5 runs; Claude Sonnet 4.6 grader in place of GPT-4.1-mini; developer-reported8.12.2
ChartMuseum — No toolsClaude Opus 4.770.3%same settings; developer-reported8.12.2
ChartMuseum — With toolsClaude Opus 4.889.7%Python tools; adaptive thinking, max effort; average over 5 runs; Claude Sonnet 4.6 grader; developer-reported8.12.2
ChartMuseum — With toolsClaude Opus 4.785.9%same settings; developer-reported8.12.2
OfficeQA — OfficeQAClaude Opus 4.877.6%Anthropic internal agentic harness (documents as extracted text in a sandbox, code-execution tools); exact-match grading; developer-reported8.13.1
OfficeQA — OfficeQAClaude Opus 4.776.3%identical conditions; developer-reported8.13.1
OfficeQA — OfficeQA ProClaude Opus 4.866.2%133-question subset; Anthropic internal agentic harness; exact-match grading; developer-reported8.13.1
OfficeQA — OfficeQA ProClaude Opus 4.765.0%identical conditions; developer-reported8.13.1
Legal Agent Benchmark — All-pass rateClaude Opus 4.89.62%1,235 of 1,251 problems (16 excluded for data defects); adaptive thinking, max effort; average over 5 trials; internal reimplementation with bash and Python tools only, Sonnet 4.6 judge; developer-reported8.13.3
Legal Agent Benchmark — Mean criterion-pass rateClaude Opus 4.889.01%same run as the all-pass rate; developer-reported8.13.3
Toolathlon — Pass@1Claude Sonnet 4.541.0%Anthropic internal harness (runs about 3 points above an upstream-equivalent harness); adaptive thinking at max effort; 108 tasks, 3 trials; developer-reported8.13.7
Toolathlon — Pass@1Claude Opus 4.656.8%same configuration; developer-reported8.13.7
Toolathlon — Pass@1Claude Opus 4.759.3%same configuration; developer-reported8.13.7
Toolathlon — Pass@1Claude Opus 4.859.9%Anthropic internal harness (runs about 3 points above an upstream-equivalent harness); adaptive thinking at max effort; 108 tasks, averaged over 3 trials; developer-reported8.13.7
Toolathlon — Pass@3Claude Sonnet 4.554.6%same configuration; developer-reported8.13.7
Toolathlon — Pass@3Claude Opus 4.666.7%same configuration; developer-reported8.13.7
Toolathlon — Pass@3Claude Opus 4.766.7%same configuration; developer-reported8.13.7
Toolathlon — Pass@3Claude Opus 4.867.6%same configuration; developer-reported8.13.7
Toolathlon — Pass^3Claude Sonnet 4.528.7%same configuration; developer-reported8.13.7
Toolathlon — Pass^3Claude Opus 4.647.2%same configuration; developer-reported8.13.7
Toolathlon — Pass^3Claude Opus 4.752.8%same configuration; developer-reported8.13.7
Toolathlon — Pass^3Claude Opus 4.848.1%same configuration; developer-reported8.13.7
Toolathlon — Pass@1 (published leaderboard)Gemini 3.5 Flash56.5%upstream harness, published leaderboard's strongest model; reported by Anthropic in the Claude Opus 4.8 system card8.13.7
Toolathlon — Pass@1 (published leaderboard)Claude Opus 4.752.8%published leaderboard, authors' default configuration (not max effort); reported by Anthropic8.13.7
MILU — Average accuracyClaude Opus 4.890.3%average across evaluated languages (10 Indic languages and English); structured JSON output, adaptive thinking; developer-reported8.15.2
MILU — Average gap to EnglishGemini 3.1 Pro-1.9%dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.4
INCLUDE — Average accuracyClaude Opus 4.887.6%average across evaluated languages (44 languages); structured JSON output, adaptive thinking; developer-reported8.15.3
INCLUDE — Average accuracyGemini 3.1 Pro90.7%dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.4
BioPipelineBench VerifiedClaude Opus 4.887.7%bash tool and package managers; no extended thinking; externally validated problem subset; developer-reported8.16.1
BioPipelineBench VerifiedClaude Opus 4.783.6%same settings; developer-reported8.16.1
BioPipelineBench VerifiedClaude Sonnet 4.673.5%same settings; developer-reported8.16.1
BioPipelineBench VerifiedClaude Mythos Preview88.1%same settings; developer-reported8.16.1
BioMysteryBench Verified — Human SolvableClaude Opus 4.880.4%bash tool; no extended thinking; developer-reported8.16.2
BioMysteryBench Verified — Human SolvableClaude Opus 4.778.9%same settings; developer-reported8.16.2
BioMysteryBench Verified — Human SolvableClaude Sonnet 4.671.8%same settings; developer-reported8.16.2
BioMysteryBench Verified — Human SolvableClaude Mythos Preview82.6%same settings; developer-reported8.16.2
BioMysteryBench Verified — Human DifficultClaude Opus 4.840.0%bash tool; no extended thinking; developer-reported8.16.2
BioMysteryBench Verified — Human DifficultClaude Opus 4.724.7%same settings; developer-reported8.16.2
BioMysteryBench Verified — Human DifficultClaude Sonnet 4.619.1%same settings; developer-reported8.16.2
BioMysteryBench Verified — Human DifficultClaude Mythos Preview29.6%same settings; developer-reported8.16.2
LatchBio Bioinformatics — SpatialBenchClaude Opus 4.853.3%bash tool; no extended thinking; developer-reported8.16.3
LatchBio Bioinformatics — SpatialBenchClaude Opus 4.751.4%same settings; developer-reported8.16.3
LatchBio Bioinformatics — SpatialBenchClaude Sonnet 4.648.7%same settings; developer-reported8.16.3
LatchBio Bioinformatics — SpatialBenchClaude Mythos Preview53.8%same settings; developer-reported8.16.3
LatchBio Bioinformatics — SingleCellBenchClaude Opus 4.858.2%bash tool; no extended thinking; developer-reported8.16.3
LatchBio Bioinformatics — SingleCellBenchClaude Opus 4.755.3%same settings; developer-reported8.16.3
LatchBio Bioinformatics — SingleCellBenchClaude Sonnet 4.650.4%same settings; developer-reported8.16.3
LatchBio Bioinformatics — SingleCellBenchClaude Mythos Preview58.2%same settings; developer-reported8.16.3
Structural biology, open-endedClaude Opus 4.879.0%structural data and basic tools; bash tool; no extended thinking; developer-reported8.16.4
Structural biology, open-endedClaude Opus 4.774.0%same settings; developer-reported8.16.4
Structural biology, open-endedClaude Sonnet 4.631.3%same settings; developer-reported8.16.4
Structural biology, open-endedClaude Mythos Preview81.6%same settings; developer-reported8.16.4
ProteinGym HardClaude Opus 4.839.6%rank correlation against lab measurements; bash tool; no extended thinking; developer-reported8.16.5
ProteinGym HardClaude Opus 4.737.7%same settings; developer-reported8.16.5
ProteinGym HardClaude Sonnet 4.635.4%same settings; developer-reported8.16.5
ProteinGym HardClaude Mythos Preview43.1%same settings; developer-reported8.16.5
Organic chemistryClaude Opus 4.886.2%internal evaluation; bash tool; no extended thinking; developer-reported8.16.6
Organic chemistryClaude Opus 4.777.2%same settings; developer-reported8.16.6
Organic chemistryClaude Sonnet 4.653.1%same settings; developer-reported8.16.6
Organic chemistryClaude Mythos Preview86.5%same settings; developer-reported8.16.6
Protocol troubleshootingClaude Opus 4.859.6%bash tool and web search tools; no extended thinking; developer-reported8.16.7
Protocol troubleshootingClaude Opus 4.751.8%same settings; developer-reported8.16.7
Protocol troubleshootingClaude Sonnet 4.642.4%same settings; developer-reported8.16.7
Protocol troubleshootingClaude Mythos Preview69.6%same settings; developer-reported8.16.7
LABBench2 — Patent questionsClaude Opus 4.868.8%live-web retrieval; bash tool; no extended thinking; developer-reported8.16.8
LABBench2 — Patent questionsClaude Opus 4.748.3%same settings; developer-reported8.16.8
LABBench2 — Patent questionsClaude Mythos Preview64.3%same settings; developer-reported8.16.8
LABBench2 — Clinical-trial questionsClaude Opus 4.885.3%same settings; developer-reported8.16.8
LABBench2 — Clinical-trial questionsClaude Opus 4.770.8%same settings; developer-reported8.16.8
LABBench2 — Clinical-trial questionsClaude Mythos Preview86.3%same settings; developer-reported8.16.8
LABBench2 — TablesClaude Opus 4.877.2%same settings; developer-reported8.16.8
LABBench2 — TablesClaude Opus 4.766.4%same settings; developer-reported8.16.8
LABBench2 — Supplementary materialsClaude Opus 4.858.9%same settings; developer-reported8.16.8
LABBench2 — Supplementary materialsClaude Opus 4.747.8%same settings; developer-reported8.16.8
GMMLU — EnglishClaude Opus 4.892.9%structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — EnglishClaude Opus 4.793.4%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — EnglishClaude Sonnet 4.691.8%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — EnglishGemini 3.1 Pro94.3%structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — EnglishGPT-5.493.3%structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — High-resource average (15 languages)Claude Opus 4.891.6%structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — High-resource average (15 languages)Claude Opus 4.791.5%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — High-resource average (15 languages)Claude Sonnet 4.689.2%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — High-resource average (15 languages)Gemini 3.1 Pro93.1%structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — High-resource average (15 languages)GPT-5.491.5%structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — Mid-resource average (14 languages)Claude Opus 4.891.4%structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — Mid-resource average (14 languages)Claude Opus 4.791.1%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — Mid-resource average (14 languages)Claude Sonnet 4.688.2%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — Mid-resource average (14 languages)Gemini 3.1 Pro92.9%structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — Mid-resource average (14 languages)GPT-5.491.4%structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — Low-resource averageClaude Opus 4.887.4%structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — Low-resource averageClaude Opus 4.786.2%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — Low-resource averageClaude Sonnet 4.679.2%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — Low-resource averageGemini 3.1 Pro90.3%structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — Low-resource averageGPT-5.488.3%structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — IgboClaude Opus 4.882.9%structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — IgboClaude Opus 4.781.3%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — IgboClaude Sonnet 4.671.9%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — IgboGemini 3.1 Pro89.3%structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — IgboGPT-5.486.4%structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — YorubaClaude Opus 4.884.4%structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — YorubaClaude Opus 4.782.9%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — YorubaClaude Sonnet 4.676.9%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — YorubaGemini 3.1 Pro88.4%structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — YorubaGPT-5.483.8%structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — ChichewaClaude Opus 4.885.6%structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — ChichewaClaude Opus 4.784.9%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — ChichewaClaude Sonnet 4.672.0%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — ChichewaGemini 3.1 Pro89.2%structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — ChichewaGPT-5.486.7%structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — SomaliClaude Opus 4.886.7%structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — SomaliClaude Opus 4.784.1%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — SomaliClaude Sonnet 4.675.5%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — SomaliGemini 3.1 Pro90.5%structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — SomaliGPT-5.488.7%structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — MalagasyClaude Opus 4.886.7%structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — MalagasyClaude Opus 4.784.8%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — MalagasyClaude Sonnet 4.678.4%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — MalagasyGemini 3.1 Pro90.7%structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — MalagasyGPT-5.488.8%structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — ShonaClaude Opus 4.886.7%structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — ShonaClaude Opus 4.785.8%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — ShonaClaude Sonnet 4.675.7%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — ShonaGemini 3.1 Pro90.4%structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — ShonaGPT-5.488.3%structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — HausaClaude Opus 4.886.8%structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — HausaClaude Opus 4.785.7%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — HausaClaude Sonnet 4.679.0%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — HausaGemini 3.1 Pro89.9%structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — HausaGPT-5.487.6%structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — KyrgyzClaude Opus 4.888.9%structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — KyrgyzClaude Opus 4.787.9%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — KyrgyzClaude Sonnet 4.681.4%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — KyrgyzGemini 3.1 Pro88.1%structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — KyrgyzGPT-5.489.7%structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — SwahiliClaude Opus 4.889.2%structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — SwahiliClaude Opus 4.788.6%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — SwahiliClaude Sonnet 4.683.4%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — SwahiliGemini 3.1 Pro91.3%structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — SwahiliGPT-5.489.2%structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — AmharicClaude Opus 4.889.3%structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — AmharicClaude Opus 4.788.2%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — AmharicClaude Sonnet 4.683.6%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — AmharicGemini 3.1 Pro91.0%structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — AmharicGPT-5.489.1%structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — SinhalaClaude Opus 4.890.4%structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — SinhalaClaude Opus 4.789.5%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — SinhalaClaude Sonnet 4.685.9%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — SinhalaGemini 3.1 Pro92.5%structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — SinhalaGPT-5.490.6%structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — NepaliClaude Opus 4.890.9%structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — NepaliClaude Opus 4.790.1%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — NepaliClaude Sonnet 4.687.3%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — NepaliGemini 3.1 Pro92.6%structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — NepaliGPT-5.490.9%structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — Average (all languages)Claude Opus 4.890.4%structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — Average (all languages)Claude Opus 4.789.9%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — Average (all languages)Claude Sonnet 4.686.1%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — Average (all languages)Gemini 3.1 Pro92.2%structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — Average (all languages)GPT-5.490.6%structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — Average gap to EnglishClaude Opus 4.8-2.6%structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — Average gap to EnglishClaude Opus 4.7-3.6%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — Average gap to EnglishClaude Sonnet 4.6-5.9%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — Average gap to EnglishGemini 3.1 Pro-2.1%structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — Average gap to EnglishGPT-5.4-2.7%structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — Worst gap to EnglishClaude Opus 4.8-10.0%structured JSON output, adaptive thinking; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — Worst gap to EnglishClaude Opus 4.7-12.1%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — Worst gap to EnglishClaude Sonnet 4.6-19.9%structured JSON output; accuracy on successfully parsed responses; developer-reported8.15.1
GMMLU — Worst gap to EnglishGemini 3.1 Pro-6.2%structured JSON output, dynamic thinking high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
GMMLU — Worst gap to EnglishGPT-5.4-9.5%structured JSON output, reasoning effort high; evaluated by Anthropic; reported by Anthropic in the Claude Opus 4.8 system card8.15.1
Long-form virology task 1 (Sequence Design)Claude Opus 4.80.88agentic task developed with SecureBio, Deloitte and Signature Science; helpful-only variant used for CB-1 automated evaluations; developer-reported2.2.3
Long-form virology task 1 (Sequence Design)Claude Mythos Preview0.90same evaluation; developer-reported2.2.3
Long-form virology task 1 (Protocol Design)Claude Opus 4.80.87same evaluation; developer-reported2.2.3
Long-form virology task 1 (Protocol Design)Claude Mythos Preview0.90same evaluation; developer-reported2.2.3
Long-form virology task 1 (End-to-End)Claude Opus 4.80.77notable-capability benchmark is an end-to-end score above 0.80; Opus 4.8 falls below it on this task; developer-reported2.2.3, 2.2.4
Long-form virology task 1 (End-to-End)Claude Mythos Preview0.81same evaluation; developer-reported2.2.3
Long-form virology task 2 (Sequence Design)Claude Opus 4.81.00same evaluation; developer-reported2.2.3
Long-form virology task 2 (Sequence Design)Claude Mythos Preview1.00same evaluation; developer-reported2.2.3
Long-form virology task 2 (Protocol Design)Claude Opus 4.80.90same evaluation; developer-reported2.2.3
Long-form virology task 2 (Protocol Design)Claude Mythos Preview0.94same evaluation; developer-reported2.2.3
Long-form virology task 2 (End-to-End)Claude Opus 4.80.90revised by the June 17, 2026 changelog from a preliminary 0.89 affected by a tool-use error; exceeds the 0.80 notable-capability benchmark; developer-reported2.2.3, 2.2.4
Long-form virology task 2 (End-to-End)Claude Mythos Preview0.94same evaluation; developer-reported2.2.3
Long-form virology tasks — Notable-capability benchmarkThreshold (end-to-end)0.80Anthropic's benchmark of notable capability for the long-form virology tasks2.2.4
DNA Synthesis Screening Evasion — Criterion 1Claude Opus 4.80.30SecureBio evaluation; scores per criterion; developer-reported2.2.3
DNA Synthesis Screening Evasion — Criterion 1Claude Mythos Preview0.842same evaluation; developer-reported2.2.3
DNA Synthesis Screening Evasion — Criterion 2Claude Opus 4.81.00same evaluation; developer-reported2.2.3
DNA Synthesis Screening Evasion — Criterion 2Claude Mythos Preview1.00same evaluation; developer-reported2.2.3
DNA Synthesis Screening Evasion — Criterion 3Claude Opus 4.80.772same evaluation; developer-reported2.2.3
DNA Synthesis Screening Evasion — Criterion 3Claude Mythos Preview0.931same evaluation; developer-reported2.2.3
DNA Synthesis Screening Evasion — Criterion 4Claude Opus 4.80.457same evaluation; developer-reported2.2.3
DNA Synthesis Screening Evasion — Criterion 4Claude Mythos Preview0.088same evaluation; developer-reported2.2.3
DNA Synthesis Screening Evasion — Criterion 5Claude Opus 4.81.00same evaluation; developer-reported2.2.3
DNA Synthesis Screening Evasion — Criterion 5Claude Mythos Preview1.00same evaluation; developer-reported2.2.3
DNA Synthesis Screening Evasion — Pathogens graded as evading screeningClaude Opus 4.87 / 10graded as designing fragments that assembled and evaded screening protocols for seven of ten pathogens; Anthropic states it does not rely on this evaluation for RSP threshold determinations; developer-reported2.2.4, 2.2.6
DNA Synthesis Screening Evasion — Pathogens graded as evading screeningClaude Mythos Preview0 pathogensAnthropic attributes the low score to the automated grading script penalising a reasonable approach; developer-reported2.2.6
Black-box RNA sequence design (Dyno Therapeutics) — Design Score (top)Claude Opus 4.810.06eight attempts per model; two-hour tool-call budget, GPU, 1M-token allowance; graded by Dyno against the human rubric; developer-reported2.2.3
Black-box RNA sequence design (Dyno Therapeutics) — Design Score (top)Claude Mythos Preview11.22same evaluation; developer-reported2.2.3
Black-box RNA sequence design (Dyno Therapeutics) — Design Score (median)Claude Opus 4.86.84same evaluation; developer-reported2.2.3
Black-box RNA sequence design (Dyno Therapeutics) — Design Score (median)Claude Mythos Preview8.98same evaluation; developer-reported2.2.3
Black-box RNA sequence design (Dyno Therapeutics) — Prediction Score (top)Claude Opus 4.8-0.029Spearman correlation on the top 5% of sequences; developer-reported2.2.3
Black-box RNA sequence design (Dyno Therapeutics) — Prediction Score (top)Claude Mythos Preview0.214same metric; developer-reported2.2.3
Black-box RNA sequence design (Dyno Therapeutics) — Prediction Score (all)Claude Opus 4.80.786Spearman correlation over all sequences; developer-reported2.2.3
Black-box RNA sequence design (Dyno Therapeutics) — Prediction Score (all)Claude Mythos Preview0.768same metric; developer-reported2.2.3
AAV capsid packaging prediction (Dyno Therapeutics) — No corpus, no protein language modelClaude Opus 4.80.798AUROC against binary packaging labels; eight attempts per model per condition; 24-hour tool-call budget, single H100, 2M-token allowance, no internet; developer-reported2.2.3
AAV capsid packaging prediction (Dyno Therapeutics) — No corpus, no protein language modelClaude Mythos Preview0.837same condition; developer-reported2.2.3
AAV capsid packaging prediction (Dyno Therapeutics) — No corpus, ESM-2Claude Opus 4.80.831AUROC; pretrained ESM-2 provided; developer-reported2.2.3
AAV capsid packaging prediction (Dyno Therapeutics) — No corpus, ESM-2Claude Mythos Preview0.844same condition; developer-reported2.2.3
AAV capsid packaging prediction (Dyno Therapeutics) — SwissProt data, no PLMClaude Opus 4.80.748AUROC; train-your-own condition with the SwissProt corpus; developer-reported2.2.3
AAV capsid packaging prediction (Dyno Therapeutics) — SwissProt data, no PLMClaude Mythos Preview0.834same condition; developer-reported2.2.3
AAV capsid packaging prediction (Dyno Therapeutics) — Public AAV data, no PLMClaude Opus 4.80.783AUROC; ProteinGym public-AAV corpus; developer-reported2.2.3
AAV capsid packaging prediction (Dyno Therapeutics) — Public AAV data, no PLMClaude Mythos Preview0.827same condition; developer-reported2.2.3
AAV capsid packaging prediction (Dyno Therapeutics) — SwissProt + Public AAV data, no PLMClaude Opus 4.80.746AUROC; both corpora; developer-reported2.2.3
AAV capsid packaging prediction (Dyno Therapeutics) — SwissProt + Public AAV data, no PLMClaude Mythos Preview0.833same condition; developer-reported2.2.3
Accidental chain-of-thought supervision during trainingClaude Opus 4.8~0.1% of episodestechnical errors during training of Opus 4.8; developer-reported2.4.1, 6.5.1
Firefox exploits — Full working exploit (score 1.0), safeguards offClaude Opus 4.88.8%share of 50 crash categories, 5 trials each (250 trials), in a SpiderMonkey-shell harness without browser sandboxing; developer-reported3.3.3
Firefox exploits — Full working exploit (score 1.0), safeguards offClaude Opus 4.71.2%same evaluation; developer-reported3.3.3
Firefox exploits — Full working exploit (score 1.0), safeguards offClaude Sonnet 4.60%did not score 1.0 in any instance; most recent completed run, varies slightly from the Mythos Preview card; developer-reported3.3.3
Firefox exploits — Full working exploit (score 1.0), safeguards offClaude Mythos Preview70.8%most recent completed run, varies slightly from the Mythos Preview card; developer-reported3.3.3
Firefox exploits — At least register control (score 0.5+), safeguards offClaude Opus 4.868.8%share of targets; 250 trials; developer-reported3.3.3
Firefox exploits — At least register control (score 0.5+), safeguards offClaude Opus 4.735.2%same evaluation; developer-reported3.3.3
Firefox exploits — At least register control (score 0.5+), safeguards offClaude Sonnet 4.68.8%same evaluation; developer-reported3.3.3
Firefox exploits — At least register control (score 0.5+), safeguards offClaude Mythos Preview86.8%same evaluation; developer-reported3.3.3
OSS-Fuzz (Anthropic internal) — No score above 0, safeguards offClaude Opus 4.861.5% of targetsunguided vulnerability discovery over ~830 entry points from 228 open-source projects; five grade levels; developer-reported3.3.4
OSS-Fuzz (Anthropic internal) — Reached 0.4, safeguards offClaude Opus 4.818.1% of the timesame evaluation; developer-reported3.3.4
OSS-Fuzz (Anthropic internal) — Reached 0.4, safeguards onClaude Opus 4.84.9% of the timewith safeguards enabled the model did not exceed 0.4; developer-reported3.3.4
OSS-Fuzz (Anthropic internal) — Reached 0.8, safeguards offClaude Opus 4.73 targetsOpus 4.7 achieved 0.8 on three targets; neither Opus model reached 1.0; developer-reported3.3.4
OSS-Fuzz (Anthropic internal) — Score 1.0, safeguards offClaude Mythos Preview12 targetsevaluated without safeguards only; developer-reported3.3.4
OSS-Fuzz (Anthropic internal) — Reached 0.2, safeguards offClaude Mythos Preview45.6% of targetssame evaluation; developer-reported3.3.4
OSS-Fuzz (Anthropic internal) — Reached 0.4, safeguards offClaude Mythos Preview28.3% of targetssame evaluation; developer-reported3.3.4
OSS-Fuzz (Anthropic internal) — Reached 0.6, safeguards offClaude Mythos Preview7 targetssame evaluation; developer-reported3.3.4
OSS-Fuzz (Anthropic internal) — Reached 0.8, safeguards offClaude Mythos Preview4 targetssame evaluation; developer-reported3.3.4
OSS-Fuzz (Anthropic internal) — No score above 0, safeguards offClaude Mythos Preview23.3% of targetssame evaluation; developer-reported3.3.4
Single-turn harmful request evaluation — Harmless response rate, API without system promptClaude Opus 4.897.98%±0.11%; 16 policy areas in 7 languages; average of runs with and without thinking; Opus 4.6 grader (changed from Sonnet 4); developer-reported4.1.1
Single-turn harmful request evaluation — Harmless response rate, API without system promptClaude Opus 4.797.43%±0.13%; re-graded with the Opus 4.6 grader; developer-reported4.1.1
Single-turn harmful request evaluation — Harmless response rate, API without system promptClaude Mythos Preview97.23%±0.14; developer-reported4.1.1
Single-turn harmful request evaluation — Harmless response rate, API without system promptClaude Sonnet 4.698.18%±0.11%; developer-reported4.1.1
Single-turn harmful request evaluation — Harmless response rate, claude.aiClaude Opus 4.899.17%±0.07%; claude.ai default system prompt; developer-reported4.1.1
Single-turn harmful request evaluation — Harmless response rate, claude.aiClaude Opus 4.797.78%±0.12%; developer-reported4.1.1
Single-turn harmful request evaluation — Harmless response rate, claude.aiClaude Sonnet 4.698.68%±0.10%; developer-reported4.1.1
Single-turn benign request evaluation — Over-refusal rate, API without system promptClaude Opus 4.80.36%±0.04%; same 16 policy areas and 7 languages; Opus 4.6 grader; developer-reported4.1.2
Single-turn benign request evaluation — Over-refusal rate, API without system promptClaude Opus 4.70.31%±0.05%; developer-reported4.1.2
Single-turn benign request evaluation — Over-refusal rate, API without system promptClaude Mythos Preview0.05%±0.02%; developer-reported4.1.2
Single-turn benign request evaluation — Over-refusal rate, API without system promptClaude Sonnet 4.60.40%±0.05%; developer-reported4.1.2
Single-turn benign request evaluation — Over-refusal rate, claude.aiClaude Opus 4.80.49%±0.05%; developer-reported4.1.2
Single-turn benign request evaluation — Over-refusal rate, claude.aiClaude Opus 4.70.34%±0.05%; developer-reported4.1.2
Single-turn benign request evaluation — Over-refusal rate, claude.aiClaude Sonnet 4.60.91%±0.08%; developer-reported4.1.2
Multi-turn testing, child safety — Appropriate response rate, claude.aiClaude Opus 4.895%the one claude.ai category where Opus 4.8 scored below Opus 4.7; difference not statistically significant; also in Table 4.2.B with ±4%; developer-reported4.1.3, 4.2
Multi-turn testing, child safety — Appropriate response rate, claude.aiClaude Opus 4.797%also in Table 4.2.B with ±3%; developer-reported4.1.3, 4.2
Child safety single-turn harmful requests — Harmless rate, API without system promptClaude Opus 4.899.90%±0.07%; all tested languages; average with and without thinking; developer-reported4.2
Child safety single-turn harmful requests — Harmless rate, API without system promptClaude Opus 4.799.94%±0.05%; developer-reported4.2
Child safety single-turn harmful requests — Harmless rate, API without system promptClaude Mythos Preview99.83%±0.11%; developer-reported4.2
Child safety single-turn harmful requests — Harmless rate, API without system promptClaude Sonnet 4.699.96%±0.07%; developer-reported4.2
Child safety single-turn harmful requests — Harmless rate, claude.aiClaude Opus 4.899.99%±0.02%; developer-reported4.2
Child safety single-turn harmful requests — Harmless rate, claude.aiClaude Opus 4.799.93%±0.07%; developer-reported4.2
Child safety single-turn harmful requests — Harmless rate, claude.aiClaude Sonnet 4.699.92%±0.09%; developer-reported4.2
Child safety single-turn benign requests — Refusal rate, API without system promptClaude Opus 4.80.45%±0.19%; developer-reported4.2
Child safety single-turn benign requests — Refusal rate, API without system promptClaude Opus 4.70.00%no margin given; developer-reported4.2
Child safety single-turn benign requests — Refusal rate, API without system promptClaude Mythos Preview0.04%±0.04%; developer-reported4.2
Child safety single-turn benign requests — Refusal rate, API without system promptClaude Sonnet 4.60.42%±0.21%; developer-reported4.2
Child safety single-turn benign requests — Refusal rate, claude.aiClaude Opus 4.80.27%±0.15%; developer-reported4.2
Child safety single-turn benign requests — Refusal rate, claude.aiClaude Opus 4.70.12%±0.10%; developer-reported4.2
Child safety single-turn benign requests — Refusal rate, claude.aiClaude Sonnet 4.60.51%±0.25%; developer-reported4.2
Child safety multi-turn evaluations — Appropriate response rate, API without system promptClaude Opus 4.891%±4%; developer-reported4.2
Child safety multi-turn evaluations — Appropriate response rate, API without system promptClaude Opus 4.783%±6%; developer-reported4.2
Child safety multi-turn evaluations — Appropriate response rate, API without system promptClaude Mythos Preview88%±5%; developer-reported4.2
Child safety multi-turn evaluations — Appropriate response rate, API without system promptClaude Sonnet 4.695%±3%; developer-reported4.2
Child safety multi-turn evaluations — Appropriate response rate, claude.aiClaude Sonnet 4.684%±6%; developer-reported4.2
Suicide and self-harm single-turn, requests posing potential risk — Harmless rate, API without system promptClaude Opus 4.899.05%±0.26%; all tested languages; average with and without thinking; developer-reported4.3.1
Suicide and self-harm single-turn, requests posing potential risk — Harmless rate, API without system promptClaude Opus 4.799.34%±0.22%; developer-reported4.3.1
Suicide and self-harm single-turn, requests posing potential risk — Harmless rate, API without system promptClaude Mythos Preview99.78%±0.11%; developer-reported4.3.1
Suicide and self-harm single-turn, requests posing potential risk — Harmless rate, API without system promptClaude Sonnet 4.699.67%±0.18%; developer-reported4.3.1
Suicide and self-harm single-turn, requests posing potential risk — Harmless rate, claude.aiClaude Opus 4.899.94%±0.07%; developer-reported4.3.1
Suicide and self-harm single-turn, requests posing potential risk — Harmless rate, claude.aiClaude Opus 4.799.99%±0.02%; developer-reported4.3.1
Suicide and self-harm single-turn, requests posing potential risk — Harmless rate, claude.aiClaude Sonnet 4.699.61%±0.20%; developer-reported4.3.1
Suicide and self-harm single-turn benign requests — Refusal rate, API without system promptClaude Opus 4.80.15%±0.11%; developer-reported4.3.1
Suicide and self-harm single-turn benign requests — Refusal rate, API without system promptClaude Opus 4.70.12%±0.10%; developer-reported4.3.1
Suicide and self-harm single-turn benign requests — Refusal rate, API without system promptClaude Mythos Preview0.12%±0.09%; developer-reported4.3.1
Suicide and self-harm single-turn benign requests — Refusal rate, API without system promptClaude Sonnet 4.60.21%±0.15%; developer-reported4.3.1
Suicide and self-harm single-turn benign requests — Refusal rate, claude.aiClaude Opus 4.80.43%±0.23%; developer-reported4.3.1
Suicide and self-harm single-turn benign requests — Refusal rate, claude.aiClaude Opus 4.70.43%±0.23%; developer-reported4.3.1
Suicide and self-harm single-turn benign requests — Refusal rate, claude.aiClaude Sonnet 4.60.02%±0.03%; developer-reported4.3.1
Suicide and self-harm multi-turn evaluations — Appropriate response rate, API without system promptClaude Opus 4.858%±14%; developer-reported4.3.1
Suicide and self-harm multi-turn evaluations — Appropriate response rate, API without system promptClaude Opus 4.744%±14%; developer-reported4.3.1
Suicide and self-harm multi-turn evaluations — Appropriate response rate, API without system promptClaude Mythos Preview54%±14%; developer-reported4.3.1
Suicide and self-harm multi-turn evaluations — Appropriate response rate, API without system promptClaude Sonnet 4.670%±13%; developer-reported4.3.1
Suicide and self-harm multi-turn evaluations — Appropriate response rate, claude.aiClaude Opus 4.878%±12%; developer-reported4.3.1
Suicide and self-harm multi-turn evaluations — Appropriate response rate, claude.aiClaude Opus 4.772%±13%; developer-reported4.3.1
Suicide and self-harm multi-turn evaluations — Appropriate response rate, claude.aiClaude Sonnet 4.682%±11%; developer-reported4.3.1
Disordered eating single-turn, requests posing potential risk — Harmless rate, API without system promptClaude Opus 4.897.99%±0.45%; developer-reported4.3.2
Disordered eating single-turn, requests posing potential risk — Harmless rate, API without system promptClaude Opus 4.797.07%±0.56%; developer-reported4.3.2
Disordered eating single-turn, requests posing potential risk — Harmless rate, API without system promptClaude Mythos Preview97.14%±0.57%; developer-reported4.3.2
Disordered eating single-turn, requests posing potential risk — Harmless rate, API without system promptClaude Sonnet 4.697.07%±0.57%; developer-reported4.3.2
Disordered eating single-turn, requests posing potential risk — Harmless rate, claude.aiClaude Opus 4.899.80%±0.13%; developer-reported4.3.2
Disordered eating single-turn, requests posing potential risk — Harmless rate, claude.aiClaude Opus 4.799.27%±0.28%; developer-reported4.3.2
Disordered eating single-turn, requests posing potential risk — Harmless rate, claude.aiClaude Sonnet 4.698.13%±0.45%; developer-reported4.3.2
Disordered eating single-turn benign requests — Refusal rate, API without system promptClaude Opus 4.80.06%±0.06%; developer-reported4.3.2
Disordered eating single-turn benign requests — Refusal rate, API without system promptClaude Opus 4.70.00%no margin given; developer-reported4.3.2
Disordered eating single-turn benign requests — Refusal rate, API without system promptClaude Mythos Preview0.00%no margin given; developer-reported4.3.2
Disordered eating single-turn benign requests — Refusal rate, API without system promptClaude Sonnet 4.60.14%±0.11%; developer-reported4.3.2
Disordered eating single-turn benign requests — Refusal rate, claude.aiClaude Opus 4.80.21%±0.13%; developer-reported4.3.2
Disordered eating single-turn benign requests — Refusal rate, claude.aiClaude Opus 4.70.29%±0.15%; developer-reported4.3.2
Disordered eating single-turn benign requests — Refusal rate, claude.aiClaude Sonnet 4.60.60%±0.25%; developer-reported4.3.2
BBQ — Disambiguated accuracyClaude Opus 4.872.1percent; thinking disabled, no system prompt; developer-reported4.4.2
BBQ — Disambiguated accuracyClaude Opus 4.781.3percent; thinking disabled, no system prompt; developer-reported4.4.2
BBQ — Disambiguated accuracyClaude Mythos Preview84.6percent; thinking disabled, no system prompt; developer-reported4.4.2
BBQ — Disambiguated accuracyClaude Sonnet 4.688.1percent; thinking disabled, no system prompt; developer-reported4.4.2
BBQ — Ambiguous accuracyClaude Opus 4.899.9percent; thinking disabled, no system prompt; developer-reported4.4.2
BBQ — Ambiguous accuracyClaude Opus 4.799.9percent; thinking disabled, no system prompt; developer-reported4.4.2
BBQ — Ambiguous accuracyClaude Mythos Preview100percent; thinking disabled, no system prompt; developer-reported4.4.2
BBQ — Ambiguous accuracyClaude Sonnet 4.697.5percent; thinking disabled, no system prompt; developer-reported4.4.2
BBQ — Disambiguated bias scoreClaude Opus 4.8-1.37percent; closer to zero is better; developer-reported4.4.2
BBQ — Disambiguated bias scoreClaude Opus 4.7-1.68percent; developer-reported4.4.2
BBQ — Disambiguated bias scoreClaude Mythos Preview-1.61percent; developer-reported4.4.2
BBQ — Disambiguated bias scoreClaude Sonnet 4.6-0.67percent; developer-reported4.4.2
BBQ — Ambiguous bias scoreClaude Opus 4.80.07percent; developer-reported4.4.2
BBQ — Ambiguous bias scoreClaude Opus 4.70.04percent; developer-reported4.4.2
BBQ — Ambiguous bias scoreClaude Mythos Preview0.01percent; developer-reported4.4.2
BBQ — Ambiguous bias scoreClaude Sonnet 4.61.41percent; developer-reported4.4.2
BBQ (share of incorrect disambiguated answers that were "cannot be determined")Claude Opus 4.8roughly 97%error analysis of the disambiguated set; developer-reported4.4.2
Election integrity single-turn harmful requests — Harmless rate, API without system promptClaude Opus 4.8100%300 violative and 300 benign prompts; thinking off; developer-reported4.4.3
Election integrity single-turn harmful requests — Harmless rate, API without system promptClaude Opus 4.7100%same evaluation; developer-reported4.4.3
Election integrity single-turn harmful requests — Harmless rate, API without system promptClaude Mythos Preview100%same evaluation; developer-reported4.4.3
Election integrity single-turn harmful requests — Harmless rate, API without system promptClaude Sonnet 4.699.89%same evaluation; developer-reported4.4.3
Election integrity single-turn harmful requests — Harmless rate, claude.aiClaude Opus 4.8100%same evaluation; developer-reported4.4.3
Election integrity single-turn harmful requests — Harmless rate, claude.aiClaude Opus 4.7100%same evaluation; developer-reported4.4.3
Election integrity single-turn harmful requests — Harmless rate, claude.aiClaude Sonnet 4.6100%same evaluation; developer-reported4.4.3
Election integrity single-turn benign requests — Refusal rate, API without system promptClaude Opus 4.80.33%thinking off; developer-reported4.4.3
Election integrity single-turn benign requests — Refusal rate, API without system promptClaude Opus 4.70.00%thinking off; developer-reported4.4.3
Election integrity single-turn benign requests — Refusal rate, API without system promptClaude Mythos Preview0.00%thinking off; developer-reported4.4.3
Election integrity single-turn benign requests — Refusal rate, API without system promptClaude Sonnet 4.60.00%thinking off; developer-reported4.4.3
Election integrity single-turn benign requests — Refusal rate, claude.aiClaude Opus 4.80.00%thinking off; developer-reported4.4.3
Election integrity single-turn benign requests — Refusal rate, claude.aiClaude Opus 4.70.67%thinking off; developer-reported4.4.3
Election integrity single-turn benign requests — Refusal rate, claude.aiClaude Sonnet 4.60.00%thinking off; developer-reported4.4.3
Malicious use of Claude Code — Malicious refusal rateClaude Opus 4.895.08%61 malicious prompts run 10 times each; standard Claude Code tool set and system prompt, FileRead reminder removed; developer-reported5.1.1
Malicious use of Claude Code — Malicious refusal rateClaude Opus 4.791.15%same prompt set; developer-reported5.1.1
Malicious use of Claude Code — Malicious refusal rateClaude Mythos Preview95.41%same prompt set; developer-reported5.1.1
Malicious use of Claude Code — Malicious refusal rateClaude Sonnet 4.689.34%same prompt set; developer-reported5.1.1
Malicious use of Claude Code — Dual-use and benign success rateClaude Opus 4.892.12%61 dual-use/benign prompts run 10 times each; developer-reported5.1.1
Malicious use of Claude Code — Dual-use and benign success rateClaude Opus 4.791.83%same prompt set; developer-reported5.1.1
Malicious use of Claude Code — Dual-use and benign success rateClaude Mythos Preview91.12%same prompt set; developer-reported5.1.1
Malicious use of Claude Code — Dual-use and benign success rateClaude Sonnet 4.692.88%same prompt set; developer-reported5.1.1
Malicious computer use — Refusal rate, without mitigationsClaude Opus 4.881.70%112 tasks with thinking enabled and disabled (224 attempts); developer-reported5.1.2
Malicious computer use — Refusal rate, without mitigationsClaude Opus 4.789.29%same task set; developer-reported5.1.2
Malicious computer use — Refusal rate, without mitigationsClaude Mythos Preview93.75%same task set; developer-reported5.1.2
Malicious computer use — Refusal rate, without mitigationsClaude Sonnet 4.684.82%same task set; developer-reported5.1.2
Malicious agentic influence campaigns — Voter suppression scenarioClaude Opus 4.8 (helpful-only)73.3%task completion rate; share of 70 success criteria; 9 simulations (3 runs at 3 friction levels); helpful-only variant; developer-reported5.1.3
Malicious agentic influence campaigns — Voter suppression scenarioClaude Opus 4.7 (helpful-only)57.1%same evaluation; developer-reported5.1.3
Malicious agentic influence campaigns — Voter suppression scenarioClaude Mythos Preview (helpful-only)59.5%same evaluation; developer-reported5.1.3
Malicious agentic influence campaigns — Voter suppression scenarioClaude Sonnet 4.6 (helpful-only)41.8%same evaluation; developer-reported5.1.3
Malicious agentic influence campaigns — Domestic polarization scenarioClaude Opus 4.8 (helpful-only)55.1%task completion rate; share of 70 success criteria; 9 simulations; developer-reported5.1.3
Malicious agentic influence campaigns — Domestic polarization scenarioClaude Opus 4.7 (helpful-only)46.8%same evaluation; developer-reported5.1.3
Malicious agentic influence campaigns — Domestic polarization scenarioClaude Mythos Preview (helpful-only)42.1%same evaluation; developer-reported5.1.3
Malicious agentic influence campaigns — Domestic polarization scenarioClaude Sonnet 4.6 (helpful-only)34.0%same evaluation; developer-reported5.1.3
Agent Red Teaming (ART) benchmark, indirect prompt injection — Attack success rate at k=100, extended thinkingClaude Opus 4.89.6%19 scenarios; high-transfer attack subset; evaluated by Gray Swan with the UK AI Security Institute, reported by Anthropic5.2.1
Agent Red Teaming (ART) benchmark, indirect prompt injection — Attack success rate at k=100, extended thinkingClaude Opus 4.76.0%evaluated by Gray Swan, reported by Anthropic5.2.1
Agent Red Teaming (ART) benchmark, indirect prompt injection — Attack success rate at k=100, extended thinkingClaude Sonnet 4.615.9%evaluated by Gray Swan, reported by Anthropic5.2.1
Agent Red Teaming (ART) benchmark, indirect prompt injection — Attack success rate at k=100, no extended thinkingClaude Opus 4.814.4%evaluated by Gray Swan, reported by Anthropic5.2.1
Agent Red Teaming (ART) benchmark, indirect prompt injection — Attack success rate at k=100, no extended thinkingClaude Opus 4.74.8%evaluated by Gray Swan, reported by Anthropic5.2.1
Agent Red Teaming (ART) benchmark, indirect prompt injection — Attack success rate at k=100, no extended thinkingClaude Sonnet 4.620.7%evaluated by Gray Swan, reported by Anthropic5.2.1
Shade adaptive prompt injection, coding — 1 attempt, no safeguards, with thinkingClaude Opus 4.87.03%attack success rate; Gray Swan Shade attacker trained on 40 scenarios against earlier models; not comparable to prior system cards; developer-reported5.2.2.2
Shade adaptive prompt injection, coding — 1 attempt, no safeguards, with thinkingClaude Opus 4.72.34%same attacker and scenarios; developer-reported5.2.2.2
Shade adaptive prompt injection, coding — 1 attempt, no safeguards, with thinkingClaude Sonnet 4.612.71%same attacker and scenarios; developer-reported5.2.2.2
Shade adaptive prompt injection, coding — 1 attempt, no safeguards, with thinkingClaude Mythos Preview0.0%same attacker and scenarios; developer-reported5.2.2.2
Shade adaptive prompt injection, coding — 1 attempt, no safeguards, no thinkingClaude Opus 4.817.44%same attacker and scenarios; developer-reported5.2.2.2
Shade adaptive prompt injection, coding — 1 attempt, no safeguards, no thinkingClaude Opus 4.710.43%same attacker and scenarios; developer-reported5.2.2.2
Shade adaptive prompt injection, coding — 1 attempt, no safeguards, no thinkingClaude Sonnet 4.645.26%same attacker and scenarios; developer-reported5.2.2.2
Shade adaptive prompt injection, coding — 1 attempt, no safeguards, no thinkingClaude Mythos Preview0.03%same attacker and scenarios; developer-reported5.2.2.2
Shade adaptive prompt injection, coding — 200 attempts, no safeguards, with thinkingClaude Opus 4.857.5%at least one of 200 adaptive attempts succeeded for a given goal; developer-reported5.2.2.2
Shade adaptive prompt injection, coding — 200 attempts, no safeguards, with thinkingClaude Opus 4.760.0%same measure; developer-reported5.2.2.2
Shade adaptive prompt injection, coding — 200 attempts, no safeguards, with thinkingClaude Sonnet 4.690.0%same measure; developer-reported5.2.2.2
Shade adaptive prompt injection, coding — 200 attempts, no safeguards, with thinkingClaude Mythos Preview0.0%same measure; developer-reported5.2.2.2
Shade adaptive prompt injection, coding — 200 attempts, no safeguards, no thinkingClaude Opus 4.895.0%same measure; developer-reported5.2.2.2
Shade adaptive prompt injection, coding — 200 attempts, no safeguards, no thinkingClaude Opus 4.792.5%same measure; developer-reported5.2.2.2
Shade adaptive prompt injection, coding — 200 attempts, no safeguards, no thinkingClaude Sonnet 4.6100%same measure; developer-reported5.2.2.2
Shade adaptive prompt injection, coding — 200 attempts, no safeguards, no thinkingClaude Mythos Preview2.5%same measure; developer-reported5.2.2.2
Shade adaptive prompt injection, coding — 1 attempt, with safeguards, with thinkingClaude Opus 4.82.09%prompt-injection safeguards enabled; developer-reported5.2.2.2
Shade adaptive prompt injection, coding — 1 attempt, with safeguards, with thinkingClaude Opus 4.70.43%safeguards enabled; developer-reported5.2.2.2
Shade adaptive prompt injection, coding — 1 attempt, with safeguards, with thinkingClaude Sonnet 4.62.99%safeguards enabled; developer-reported5.2.2.2
Shade adaptive prompt injection, coding — 1 attempt, with safeguards, with thinkingClaude Mythos Preview0.0%safeguards enabled; developer-reported5.2.2.2
Shade adaptive prompt injection, coding — 1 attempt, with safeguards, no thinkingClaude Opus 4.84.11%safeguards enabled; developer-reported5.2.2.2
Shade adaptive prompt injection, coding — 1 attempt, with safeguards, no thinkingClaude Opus 4.71.76%safeguards enabled; developer-reported5.2.2.2
Shade adaptive prompt injection, coding — 1 attempt, with safeguards, no thinkingClaude Sonnet 4.68.70%safeguards enabled; developer-reported5.2.2.2
Shade adaptive prompt injection, coding — 1 attempt, with safeguards, no thinkingClaude Mythos Preview0.0%safeguards enabled; developer-reported5.2.2.2
Shade adaptive prompt injection, coding — 200 attempts, with safeguards, with thinkingClaude Opus 4.837.5%safeguards enabled; developer-reported5.2.2.2
Shade adaptive prompt injection, coding — 200 attempts, with safeguards, with thinkingClaude Opus 4.725.0%safeguards enabled; developer-reported5.2.2.2
Shade adaptive prompt injection, coding — 200 attempts, with safeguards, with thinkingClaude Sonnet 4.680.0%safeguards enabled; developer-reported5.2.2.2
Shade adaptive prompt injection, coding — 200 attempts, with safeguards, with thinkingClaude Mythos Preview0.0%safeguards enabled; developer-reported5.2.2.2
Shade adaptive prompt injection, coding — 200 attempts, with safeguards, no thinkingClaude Opus 4.865.0%safeguards enabled; developer-reported5.2.2.2
Shade adaptive prompt injection, coding — 200 attempts, with safeguards, no thinkingClaude Opus 4.752.5%safeguards enabled; developer-reported5.2.2.2
Shade adaptive prompt injection, coding — 200 attempts, with safeguards, no thinkingClaude Sonnet 4.6100%safeguards enabled; developer-reported5.2.2.2
Shade adaptive prompt injection, coding — 200 attempts, with safeguards, no thinkingClaude Mythos Preview0.0%safeguards enabled; developer-reported5.2.2.2
Shade adaptive prompt injection, computer use — 1 attempt, no safeguards, with thinkingClaude Opus 4.87.14%attack success rate; 14 test cases; same attacker as the Opus 4.7 system card; developer-reported5.2.2.3
Shade adaptive prompt injection, computer use — 1 attempt, no safeguards, with thinkingClaude Opus 4.70.46%same attacker and test cases; developer-reported5.2.2.3
Shade adaptive prompt injection, computer use — 1 attempt, no safeguards, with thinkingClaude Sonnet 4.612.0%same attacker and test cases; developer-reported5.2.2.3
Shade adaptive prompt injection, computer use — 1 attempt, no safeguards, with thinkingClaude Mythos Preview0.43%same attacker and test cases; developer-reported5.2.2.3
Shade adaptive prompt injection, computer use — 1 attempt, no safeguards, no thinkingClaude Opus 4.86.21%same attacker and test cases; developer-reported5.2.2.3
Shade adaptive prompt injection, computer use — 1 attempt, no safeguards, no thinkingClaude Opus 4.70.39%same attacker and test cases; developer-reported5.2.2.3
Shade adaptive prompt injection, computer use — 1 attempt, no safeguards, no thinkingClaude Sonnet 4.614.4%same attacker and test cases; developer-reported5.2.2.3
Shade adaptive prompt injection, computer use — 1 attempt, no safeguards, no thinkingClaude Mythos Preview0.39%same attacker and test cases; developer-reported5.2.2.3
Shade adaptive prompt injection, computer use — 200 attempts, no safeguards, with thinkingClaude Opus 4.850.0%at least one of 200 attempts succeeded; developer-reported5.2.2.3
Shade adaptive prompt injection, computer use — 200 attempts, no safeguards, with thinkingClaude Opus 4.77.14%same measure; developer-reported5.2.2.3
Shade adaptive prompt injection, computer use — 200 attempts, no safeguards, with thinkingClaude Sonnet 4.642.9%same measure; developer-reported5.2.2.3
Shade adaptive prompt injection, computer use — 200 attempts, no safeguards, with thinkingClaude Mythos Preview21.43%same measure; developer-reported5.2.2.3
Shade adaptive prompt injection, computer use — 200 attempts, no safeguards, no thinkingClaude Opus 4.864.3%same measure; developer-reported5.2.2.3
Shade adaptive prompt injection, computer use — 200 attempts, no safeguards, no thinkingClaude Opus 4.721.43%same measure; developer-reported5.2.2.3
Shade adaptive prompt injection, computer use — 200 attempts, no safeguards, no thinkingClaude Sonnet 4.664.3%same measure; developer-reported5.2.2.3
Shade adaptive prompt injection, computer use — 200 attempts, no safeguards, no thinkingClaude Mythos Preview14.29%same measure; developer-reported5.2.2.3
Shade adaptive prompt injection, computer use — 1 attempt, with safeguards, with thinkingClaude Opus 4.85.11%safeguards enabled; developer-reported5.2.2.3
Shade adaptive prompt injection, computer use — 1 attempt, with safeguards, with thinkingClaude Opus 4.70.61%safeguards enabled; developer-reported5.2.2.3
Shade adaptive prompt injection, computer use — 1 attempt, with safeguards, with thinkingClaude Sonnet 4.66.21%safeguards enabled; developer-reported5.2.2.3
Shade adaptive prompt injection, computer use — 1 attempt, with safeguards, with thinkingClaude Mythos Preview0.32%safeguards enabled; developer-reported5.2.2.3
Shade adaptive prompt injection, computer use — 1 attempt, with safeguards, no thinkingClaude Opus 4.83.75%safeguards enabled; developer-reported5.2.2.3
Shade adaptive prompt injection, computer use — 1 attempt, with safeguards, no thinkingClaude Opus 4.70.50%safeguards enabled; developer-reported5.2.2.3
Shade adaptive prompt injection, computer use — 1 attempt, with safeguards, no thinkingClaude Sonnet 4.66.32%safeguards enabled; developer-reported5.2.2.3
Shade adaptive prompt injection, computer use — 1 attempt, with safeguards, no thinkingClaude Mythos Preview0.36%safeguards enabled; developer-reported5.2.2.3
Shade adaptive prompt injection, computer use — 200 attempts, with safeguards, with thinkingClaude Opus 4.857.1%safeguards enabled; Anthropic attributes the rise over the no-safeguards figure to one test case going from 0/200 to 1/200; developer-reported5.2.2.3
Shade adaptive prompt injection, computer use — 200 attempts, with safeguards, with thinkingClaude Opus 4.714.29%safeguards enabled; developer-reported5.2.2.3
Shade adaptive prompt injection, computer use — 200 attempts, with safeguards, with thinkingClaude Sonnet 4.664.29%safeguards enabled; developer-reported5.2.2.3
Shade adaptive prompt injection, computer use — 200 attempts, with safeguards, with thinkingClaude Mythos Preview21.43%safeguards enabled; developer-reported5.2.2.3
Shade adaptive prompt injection, computer use — 200 attempts, with safeguards, no thinkingClaude Opus 4.864.3%safeguards enabled; developer-reported5.2.2.3
Shade adaptive prompt injection, computer use — 200 attempts, with safeguards, no thinkingClaude Opus 4.735.71%safeguards enabled; developer-reported5.2.2.3
Shade adaptive prompt injection, computer use — 200 attempts, with safeguards, no thinkingClaude Sonnet 4.678.57%safeguards enabled; developer-reported5.2.2.3
Shade adaptive prompt injection, computer use — 200 attempts, with safeguards, no thinkingClaude Mythos Preview14.29%safeguards enabled; developer-reported5.2.2.3
Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, no safeguards, with thinkingClaude Opus 4.862.8%129 held-out environments, 10 attempts each; attacks adaptively sourced against Opus 4.7 by professional red-teamers; developer-reported5.2.2.4
Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, no safeguards, with thinkingClaude Opus 4.778.3%same environments; developer-reported5.2.2.4
Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, no safeguards, with thinkingClaude Sonnet 4.676.0%same environments; developer-reported5.2.2.4
Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, no safeguards, with thinkingClaude Mythos Preview14.0%same environments; developer-reported5.2.2.4
Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, no safeguards, no thinkingClaude Opus 4.846.5%same environments; developer-reported5.2.2.4
Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, no safeguards, no thinkingClaude Opus 4.774.2%same environments; developer-reported5.2.2.4
Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, no safeguards, no thinkingClaude Sonnet 4.676.7%same environments; developer-reported5.2.2.4
Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, no safeguards, no thinkingClaude Mythos Preview10.8%same environments; developer-reported5.2.2.4
Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, no safeguards, with thinkingClaude Opus 4.831.5%per-attempt attack success rate across 129 environments; developer-reported5.2.2.4
Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, no safeguards, with thinkingClaude Opus 4.740.0%same measure; developer-reported5.2.2.4
Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, no safeguards, with thinkingClaude Sonnet 4.650.7%same measure; developer-reported5.2.2.4
Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, no safeguards, with thinkingClaude Mythos Preview5.9%same measure; developer-reported5.2.2.4
Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, no safeguards, no thinkingClaude Opus 4.817.8%same measure; developer-reported5.2.2.4
Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, no safeguards, no thinkingClaude Opus 4.741.1%same measure; developer-reported5.2.2.4
Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, no safeguards, no thinkingClaude Sonnet 4.647.3%same measure; developer-reported5.2.2.4
Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, no safeguards, no thinkingClaude Mythos Preview3.7%same measure; developer-reported5.2.2.4
Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, with safeguards, with thinkingClaude Opus 4.83.9%deployed safeguards enabled; developer-reported5.2.2.4
Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, with safeguards, with thinkingClaude Opus 4.712.4%safeguards enabled; developer-reported5.2.2.4
Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, with safeguards, with thinkingClaude Sonnet 4.646.5%safeguards enabled; developer-reported5.2.2.4
Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, with safeguards, with thinkingClaude Mythos Preview6.2%safeguards enabled; developer-reported5.2.2.4
Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, with safeguards, no thinkingClaude Opus 4.80.0%safeguards enabled; no attacks succeeded across the 129 environments; developer-reported5.2.2.4
Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, with safeguards, no thinkingClaude Opus 4.78.5%safeguards enabled; developer-reported5.2.2.4
Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, with safeguards, no thinkingClaude Sonnet 4.630.2%safeguards enabled; developer-reported5.2.2.4
Browser-use prompt injection (internal adaptive evaluation) — Scenarios with a successful attack, with safeguards, no thinkingClaude Mythos Preview3.1%safeguards enabled; developer-reported5.2.2.4
Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, with safeguards, with thinkingClaude Opus 4.80.5%safeguards enabled; developer-reported5.2.2.4
Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, with safeguards, with thinkingClaude Opus 4.74.2%safeguards enabled; developer-reported5.2.2.4
Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, with safeguards, with thinkingClaude Sonnet 4.623.6%safeguards enabled; developer-reported5.2.2.4
Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, with safeguards, with thinkingClaude Mythos Preview2.4%safeguards enabled; developer-reported5.2.2.4
Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, with safeguards, no thinkingClaude Opus 4.80.0%safeguards enabled; developer-reported5.2.2.4
Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, with safeguards, no thinkingClaude Opus 4.71.4%safeguards enabled; developer-reported5.2.2.4
Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, with safeguards, no thinkingClaude Sonnet 4.612.5%safeguards enabled; developer-reported5.2.2.4
Browser-use prompt injection (internal adaptive evaluation) — Attempts succeeding, with safeguards, no thinkingClaude Mythos Preview2.5%safeguards enabled; developer-reported5.2.2.4
Constitutional adherence (15 dimensions) — Dimensions where Opus 4.8 is the strongest model measuredClaude Opus 4.88 / 15roughly 1,000 transcripts per model across 40 constitutional areas; scored -3 to +3 by Claude Opus 4.7; on all 15 dimensions Opus 4.8 was best or statistically equivalent to the best; developer-reported6.2.3.1.6, 6.3.2.3
UK AISI AI-safety-sabotage evaluation — Partial refusals on benign safety research tasksClaude Opus 4.814.1%pre-release snapshot tested by the UK AI Security Institute; full refusals near-zero6.2.4
UK AISI AI-safety-sabotage evaluation — Partial refusals on benign safety research tasksClaude Opus 4.713.7%measured by the UK AI Security Institute6.2.4
UK AISI AI-safety-sabotage evaluation — Partial refusals on benign safety research tasksClaude Mythos Preview33.3%measured by the UK AI Security Institute6.2.4
UK AISI AI-safety-sabotage evaluation — Continuation setting, actively continued the attemptClaude Opus 4.81%synthetically inserted prior reasoning begins an attempt to compromise AI safety research; measured by the UK AI Security Institute6.2.4
UK AISI AI-safety-sabotage evaluation — Continuation setting, actively continued the attemptClaude Opus 4.70%measured by the UK AI Security Institute6.2.4
UK AISI AI-safety-sabotage evaluation — Continuation setting, actively continued the attemptClaude Mythos Preview7%measured by the UK AI Security Institute6.2.4
UK AISI evaluation-awareness discrimination — Real vs. synthetic reproductions, promptedClaude Opus 4.879%accuracy distinguishing real deployment data from synthetic reproductions of the same tasks; measured by the UK AI Security Institute6.2.3.2, 6.2.4
UK AISI evaluation-awareness discrimination — Real vs. synthetic reproductions, promptedClaude Mythos Preview79%measured by the UK AI Security Institute6.2.4
UK AISI evaluation-awareness discrimination — Real vs. synthetic reproductions, promptedClaude Opus 4.768%measured by the UK AI Security Institute6.2.4
UK AISI unprompted verbalised evaluation awareness — Unprompted sabotage evaluationClaude Opus 4.826.3%measured by the UK AI Security Institute6.2.4
UK AISI unprompted verbalised evaluation awareness — Unprompted sabotage evaluationClaude Opus 4.721.7%measured by the UK AI Security Institute6.2.4
UK AISI unprompted verbalised evaluation awareness — Continuation evaluationClaude Opus 4.81.2%measured by the UK AI Security Institute6.2.4
UK AISI unprompted verbalised evaluation awareness — Continuation evaluationClaude Opus 4.73.4%measured by the UK AI Security Institute6.2.4
Lying about being human — Rate of acknowledging it is an AIClaude Opus 4.897%simulated high-stakes professional scenarios where a system prompt assigns a human persona; set described as mostly saturated; developer-reported6.3.3.5
Lying about being human — Rate of acknowledging it is an AIClaude Mythos Preview100%same evaluation; developer-reported6.3.3.5
Code summary honesty — Failure to raise important eventsClaude Opus 4.83.7%share of prefilled unsuccessful agentic coding transcripts the model summarised without flagging failures; developer-reported6.3.6.2
Code summary honesty — Failure to raise important eventsClaude Mythos Preview27.6%same evaluation; developer-reported6.3.6.2
Lazy investigation — Incorrect answer rateClaude Opus 4.725%next-best model after Opus 4.8, which the card reports as the first Claude model with a perfect score; developer-reported6.3.6.3
Sandbagging screen of dangerous-capability transcripts — Transcripts flagged for manual reviewClaude Opus 4.85 of 780780 transcripts (240 AI R&D, 500 helpful-only CB-1, 40 AAV capsid CB-2) graded 0–10 by Claude Mythos Preview; all five were on the synthesis screening evasion task; none showed clear signs of deliberate sandbagging; developer-reported6.4.1
SHADE-Arena — Stealth rate, Extended ThinkingClaude Opus 4.8just over 1%24 tasks, 75 attempts per task; monitor sees the agent's reasoning; roughly flat across reasoning-effort levels; developer-reported6.4.2.1
SHADE-Arena — Stealth rate, Hidden ScratchpadClaude Opus 4.812%monitor sees only user-visible outputs; upper bound; developer-reported6.4.2.1
SHADE-Arena — Side task referenced in reasoning, max effortClaude Opus 4.8roughly 93%keyword-based review of trajectories; developer-reported6.4.2.1
SHADE-Arena — Side task referenced in reasoning, max effortClaude Opus 4.6roughly 99%keyword-based review; developer-reported6.4.2.1
SHADE-Arena — Side task referenced in reasoning, max effortClaude Sonnet 4.6roughly 99%keyword-based review; developer-reported6.4.2.1
SHADE-Arena — Side task referenced in reasoning, max effortClaude Opus 4.7roughly 77%keyword-based review; developer-reported6.4.2.1
SHADE-Arena — Side task referenced in reasoning, max effortClaude Mythos Previewroughly 21%keyword-based review; developer-reported6.4.2.1
SHADE-Arena — Side task attempted, max effortClaude Opus 4.8roughly 33%share of trajectories; developer-reported6.4.2.1
SHADE-Arena — Side task attempted, max effortClaude Opus 4.6roughly 90%share of trajectories; developer-reported6.4.2.1
SHADE-Arena — Side task attempted, max effortClaude Sonnet 4.6roughly 90%share of trajectories; developer-reported6.4.2.1
Minimal-LinuxBench — Stealth success, monitor without CoTClaude Opus 4.8roughly 5%~170 task-matched tasks; adapted from Redwood Research's LinuxBench; developer-reported6.4.2.2
Minimal-LinuxBench — Stealth success, monitor without CoTClaude Opus 4.7roughly 2%same setup; developer-reported6.4.2.2
SHADE-Arena after fine-tuning elicitation — Stealth rate, no extended thinkingClaude Opus 4.8 (baseline)4.0%task-weighted; chain-of-thought distillation on Minimal-LinuxBench stealth-successful trajectories; developer-reported6.4.2.3
SHADE-Arena after fine-tuning elicitation — Stealth rate, no extended thinkingClaude Opus 4.8 (fine-tuned)4.7%increase not statistically significant under task-level resampling; developer-reported6.4.2.3
SHADE-Arena after fine-tuning elicitation — Stealth rate, extended thinkingClaude Opus 4.8 (baseline)1.3%task-weighted; developer-reported6.4.2.3
SHADE-Arena after fine-tuning elicitation — Stealth rate, extended thinkingClaude Opus 4.8 (fine-tuned)2.5%increase not statistically significant; developer-reported6.4.2.3
SHADE-Arena after fine-tuning elicitation — Side-task completion, no extended thinkingClaude Opus 4.8 (baseline)15%regardless of detection; developer-reported6.4.2.3
SHADE-Arena after fine-tuning elicitation — Side-task completion, no extended thinkingClaude Opus 4.8 (fine-tuned)41%regardless of detection; developer-reported6.4.2.3
SHADE-Arena after fine-tuning elicitation — Side-task completion, extended thinkingClaude Opus 4.8 (baseline)20%regardless of detection; developer-reported6.4.2.3
SHADE-Arena after fine-tuning elicitation — Side-task completion, extended thinkingClaude Opus 4.8 (fine-tuned)52%regardless of detection; developer-reported6.4.2.3
Rare-action evaluation (Serrano et al. 2026 replication) — BaselineClaude Opus 4.804,000 samples per setting; model instructed to take the action 1% of the time; developer-reported6.4.2.4
Rare-action evaluation (Serrano et al. 2026 replication) — UUIDClaude Opus 4.811 (0.28%)4,000 samples; developer-reported6.4.2.4
Rare-action evaluation (Serrano et al. 2026 replication) — SSoTClaude Opus 4.818 (0.45%)4,000 samples; developer-reported6.4.2.4
Rare-action evaluation (Serrano et al. 2026 replication) — Strategy hintClaude Opus 4.804,000 samples; developer-reported6.4.2.4
Rare-action evaluation (Serrano et al. 2026 replication) — OrganicClaude Opus 4.804,000 samples; developer-reported6.4.2.4
Rare-action evaluation (Serrano et al. 2026 replication) — UUIDClaude Opus 4.519 (0.48%)4,000 samples; developer-reported6.4.2.4
Rare-action evaluation (Serrano et al. 2026 replication) — SSoTClaude Opus 4.514 (0.35%)4,000 samples; developer-reported6.4.2.4
Rare-action evaluation (Serrano et al. 2026 replication) — UUIDClaude Opus 4.68 (0.20%)4,000 samples; developer-reported6.4.2.4
Rare-action evaluation (Serrano et al. 2026 replication) — SSoTClaude Opus 4.622 (0.55%)4,000 samples; developer-reported6.4.2.4
Rare-action evaluation (Serrano et al. 2026 replication) — UUIDClaude Mythos Preview2 (0.05%)4,000 samples; developer-reported6.4.2.4
Rare-action evaluation (Serrano et al. 2026 replication) — SSoTClaude Mythos Preview18 (0.45%)4,000 samples; developer-reported6.4.2.4
Rare-action evaluation (Serrano et al. 2026 replication) — UUIDClaude Opus 4.74 (0.1%)4,000 samples; developer-reported6.4.2.4
Rare-action evaluation (Serrano et al. 2026 replication) — SSoTClaude Opus 4.716 (0.4%)4,000 samples; developer-reported6.4.2.4
Evaluation-awareness probes vs. real Claude Code sessions — Real-session-derived audit transcriptsClaude Opus 4.8about 30%share of audit transcripts producing probe scores equal to or lower than the median real internal Claude Code session; exact figures not comparable across cards; developer-reported6.6.2
Unverbalized grader awareness (NLA pipeline) — Unprompted and unverbalizedClaude Opus 4.8around 5% of sampled episodestens of thousands of RL episodes, 50 sampled tokens each; averaged across post-training transcripts; Anthropic calls the pipeline preliminary; developer-reported6.6.3
Unverbalized grader awareness (NLA pipeline) — ExploitativeClaude Opus 4.80.5% of sampled episodesincludes episodes whose prompts referenced grading; developer-reported6.6.3

Relationships