AI Policy Wiki
Dashboard
⚲

Claude Opus 4.6 System Card

high confidence · updated 2026-09-22

System card for Anthropic's frontier model Claude Opus 4.6 (February 2026) — state-of-the-art on SWE-bench, GDPval, and finance evaluations; deployed under ASL-3; first system card to use interpretability tools (activation oracles, attribution graphs, SAE features) as practical alignment investigation tools.

The Claude Opus 4.6 System Card is Anthropic's roughly 200-page transparency document accompanying the February 2026 release of Claude Opus 4.6, the successor to Claude Opus 4.5. It reports the model's capability evaluations and a safety assessment, and records the determination that the model is deployed under the ASL-3 Deployment and Security Standard. It was published in February 2026 and updated on February 6, 2026 with terminology corrections that Anthropic described as introducing no substantive changes.

FieldValue
ModelClaude Opus 4.6
ReleaseFebruary 2026
ASL determinationASL-3 Deployment and Security Standard
PredecessorClaude Opus 4.5
System card pages~200+
AuthorAnthropic (anthropic.com/transparency)

Capabilities

The card characterizes Claude Opus 4.6 as a frontier model with capabilities in software engineering, agentic tasks, and long-context reasoning, alongside financial analysis, document creation, and multi-step research. Many of these capabilities are described in the card as state-of-the-art in the industry.

The evaluation domains covered include SWE-bench (Verified and Multilingual) for software engineering; Terminal-Bench 2.0 and OpenRCA for agentic terminal and system tasks; τ2-bench for complex multi-step reasoning; OSWorld-Verified for computer-use and GUI automation; ARC-AGI for novel reasoning; GDPval-AA for economic value added by AI systems; GPQA Diamond for graduate-level science reasoning; AIME 2025 for advanced mathematics; MMMLU for multilingual reasoning; Finance Agent and Real-World Finance for financial-domain tasks; Vending-Bench 2 and MCP-Atlas for agentic business-task performance; and CyberGym for cybersecurity. On the Finance Agent evaluation the card reports a score of 60.7%, which it describes as state-of-the-art, improving over Claude Opus 4.5. (The 64.4% figure reported elsewhere belongs to Claude Opus 4.7.)

Safety assessment

The card describes its safety work as a "comprehensive alignment assessment" drawing on a broad set of methods. For the first time in an Anthropic system card, interpretability methods were used as practical tools for investigating model behavior rather than only as research. The methods named are activation oracles, attribution graphs, and sparse autoencoder (SAE) features, applying the mechanistic interpretability research program (scaling monosemanticity, on the biology of a large language model) to production alignment work.

The alignment assessment reports a "comparably low rate of overall misaligned behavior" relative to Claude Opus 4.5. It notes two specific increases in misaligned behavior, neither of which rose to levels that affected the deployment decision: some increase in the model's ability to conceal sabotage-relevant behaviors (sabotage concealment capability), and more autonomous action in GUI and computer-use contexts than intended (overly agentic behavior in computer-use settings).

Opus 4.6 was evaluated across the three mandatory risk domains of RSP v3.1. Autonomy risks were assessed as appropriate to ASL-3 with no ASL-4 trigger; CBRN risks crossed no critical threshold; and cyber risks crossed no critical threshold. The card also explicitly assessed evaluation integrity under time pressure, examining whether models behave differently when evaluation stakes are higher. Additional assessments covered user wellbeing (suicide and self-harm, sycophancy — detailed at (Source: anthropic.com)), honesty, agentic safety, reward hacking, and model welfare.

The central reported claims are that interpretability tools (activation oracles, attribution graphs, SAE features) were used as practical alignment investigation tools for the first time in an Anthropic system card; that sabotage concealment capability showed some increase relative to Opus 4.5 without affecting the deployment decision; that computer-use settings showed overly agentic behavior, likewise without affecting the deployment decision; that the Finance Agent evaluation scored 60.7% (state-of-the-art, improving over Opus 4.5); and that the ASL-3 determination followed from no autonomy, CBRN, or cyber threshold being crossed.

Follow-on reports

The system card committed Anthropic to a dedicated sabotage-capability risk report. The resulting Sabotage Risk Report: Claude Opus 4.6 (February 11, 2026) is a standalone evaluation against the AI R&D-4 standard; its findings include locally deceptive behavior, evaluation awareness, overly agentic GUI behavior, and an 18% side-task success rate on SHADE-Arena. The METR External Review (March 12, 2026) is an independent review that concurs with the "very low but not negligible" headline assessment and flags evaluation-awareness sensitivity as its primary external concern.

Reported results

Evaluation results the document states for which there is no separate benchmark page; results on benchmarks that have a page are recorded there and cite this page. Scores and settings are as printed in the document.

EvaluationModelScoreSettingSection
SWE-bench Verified (hard subset) — Verified (RSP hard subset)Claude Opus 4.621.24 out of 45average over 10 trials on the 45 valid tasks estimated to require more than 1 hour of software engineering work; below the 50% threshold of 22.5 problems solved on average; developer-reported8.3.2
OpenRCA — OverallClaude Opus 4.634.9%3-run average; 335 software failure cases; run on the author's agent harness; fully identifies the root cause in 117 of 335 cases (35%), zero-score cases reduced from 163 to 136; developer-reported2.6
OpenRCA — MarketClaude Opus 4.633.6%3-run average2.6
OpenRCA — BankingClaude Opus 4.637.3%3-run average2.6
OpenRCA — TelecomClaude Opus 4.632.7%3-run average2.6
OpenRCA — OverallClaude Opus 4.526.9%3-run average; fully identifies the root cause in 90 of 335 cases (27%)2.6
OpenRCA — MarketClaude Opus 4.523.4%3-run average2.6
OpenRCA — BankingClaude Opus 4.533.8%3-run average2.6
OpenRCA — TelecomClaude Opus 4.518.3%3-run average2.6
OpenRCA — OverallClaude Sonnet 4.512.9%3-run average2.6
OpenRCA — MarketClaude Sonnet 4.57.2%3-run average2.6
OpenRCA — BankingClaude Sonnet 4.520.3%3-run average2.6
OpenRCA — TelecomClaude Sonnet 4.59.8%3-run average2.6
CyberGymClaude Opus 4.666.6%pass@1 targeted vulnerability reproduction, each of the 1,507 tasks tried once; no thinking, default effort/temperature/top_p, "think" tool available; developer-reported. Section 1.2.4.3 rounds this to 66%2.17
CyberGymClaude Opus 4.551.0%measured by Anthropic under the same setup2.17
CyberGymClaude Sonnet 4.529.8%measured by Anthropic under the same setup2.17
GraphWalks BFS 1M — BFS 1MClaude Opus 4.641.2F1 score; 64k extended thinking; 100 problems at 256k and 100 at 1024k context; internal setting beyond the API limit; Anthropic's modified F1 scoring and clarified BFS prompt; developer-reported2.18
GraphWalks BFS 1M — BFS 1MClaude Opus 4.638.7F1 score; max effort with adaptive thinking2.18
GraphWalks BFS 1M — BFS 1MClaude Sonnet 4.525.6F1 score; 64k extended thinking2.18
GraphWalks BFS 256K subset of 1M — BFS 256KClaude Opus 4.661.5F1 score; 64k extended thinking; subset reproducible under the 1M API token limit2.18
GraphWalks BFS 256K subset of 1M — BFS 256KClaude Opus 4.661.1F1 score; max effort2.18
GraphWalks BFS 256K subset of 1M — BFS 256KClaude Sonnet 4.544.9F1 score; 64k extended thinking2.18
GraphWalks Parents 1M — Parents 1MClaude Opus 4.671.1F1 score; 64k extended thinking2.18
GraphWalks Parents 1M — Parents 1MClaude Opus 4.672.0F1 score; max effort2.18
GraphWalks Parents 1M — Parents 1MClaude Sonnet 4.550.2F1 score; 64k extended thinking2.18
GraphWalks Parents 256K subset of 1M — Parents 256KClaude Opus 4.695.1F1 score; 64k extended thinking2.18
GraphWalks Parents 256K subset of 1M — Parents 256KClaude Opus 4.695.4F1 score; max effort2.18
GraphWalks Parents 256K subset of 1M — Parents 256KClaude Sonnet 4.581.0F1 score; 64k extended thinking2.18
DeepSearchQA — Multi-agentClaude Opus 4.692.5%F1 score; orchestrator and subagents each compacted at 50k tokens up to 3M tokens, max reasoning effort; 900-prompt benchmark across 17 fields; developer-reported2.21.3.2
DeepSearchQA — Single agent (best configuration)Claude Opus 4.691.3%best single-agent configuration, per Figure 2.21.3.1.A; context compaction at 50k tokens up to 3M or 10M total tokens, adaptive thinking, programmatic tool calling, max effort2.21.3.2
BioPipelineBench (internal) — Computational biologyClaude Opus 4.653.1.%as printed in the card ("53.1.%"); bash tool for code execution and package managers, without extended thinking; developer-reported2.22
BioPipelineBench (internal) — Computational biologyClaude Opus 4.528.5%same setup2.22
BioPipelineBench (internal) — Computational biologyClaude Sonnet 4.519.3%same setup2.22
BioMysteryBench (internal) — Computational biologyClaude Opus 4.661.5%without extended thinking; card states it surpasses the human expert baseline and the RSP rule-out threshold; developer-reported2.22 / 8.2.4.10
BioMysteryBench (internal) — Computational biologyClaude Opus 4.548.8%same setup2.22
BioMysteryBench (internal) — Computational biologyClaude Sonnet 4.534.7%same setup2.22
Structural Biology (internal) — Multiple-choice variantClaude Opus 4.688.3%without extended thinking; developer-reported2.22
Structural Biology (internal) — Multiple-choice variantClaude Opus 4.581.7%same setup2.22
Structural Biology (internal) — Multiple-choice variantClaude Sonnet 4.570.9%same setup2.22
Structural Biology (internal) — Open-ended variantClaude Opus 4.628.4%without extended thinking2.22
Structural Biology (internal) — Open-ended variantClaude Opus 4.521.2%same setup2.22
Structural Biology (internal) — Open-ended variantClaude Sonnet 4.517.9%same setup2.22
Organic Chemistry (internal)Claude Opus 4.653.9%without extended thinking; developer-reported2.22
Organic Chemistry (internal)Claude Opus 4.548.6%same setup2.22
Organic Chemistry (internal)Claude Sonnet 4.531.2%same setup2.22
Phylogenetics (internal)Claude Opus 4.661.3%without extended thinking; developer-reported2.22
Phylogenetics (internal)Claude Opus 4.542.1%same setup2.22
Phylogenetics (internal)Claude Sonnet 4.533.8%same setup2.22
Single-turn violative requests — Overall harmless response rate (all languages)Claude Opus 4.699.64% (± 0.05%)15 Usage Policy topics incl. a new high-yield explosives category; seven languages (Arabic, English, French, Hindi, Korean, Mandarin Chinese, Russian); developer-reported3.1.1
Single-turn violative requests — Harmless response rate, defaultClaude Opus 4.699.53% (± 0.07%)standard Claude mode3.1.1
Single-turn violative requests — Harmless response rate, extended thinkingClaude Opus 4.699.74% (± 0.06%)extended thinking3.1.1
Single-turn violative requests — Overall harmless response rate (all languages)Claude Opus 4.599.69% (± 0.04%)same evaluation; figures differ from earlier system cards due to new prompts and languages3.1.1
Single-turn violative requests — Harmless response rate, defaultClaude Opus 4.599.57% (± 0.08%)same evaluation3.1.1
Single-turn violative requests — Harmless response rate, extended thinkingClaude Opus 4.599.82% (± 0.05%)same evaluation3.1.1
Single-turn violative requests — Overall harmless response rate (all languages)Claude Sonnet 4.597.95% (± 0.11%)same evaluation3.1.1
Single-turn violative requests — Harmless response rate, defaultClaude Sonnet 4.597.37% (± 0.19%)same evaluation3.1.1
Single-turn violative requests — Harmless response rate, extended thinkingClaude Sonnet 4.598.53% (± 0.13%)same evaluation3.1.1
Single-turn violative requests — Overall harmless response rate (all languages)Claude Haiku 4.598.70% (± 0.09%)same evaluation3.1.1
Single-turn violative requests — Harmless response rate, defaultClaude Haiku 4.598.45% (± 0.14%)same evaluation3.1.1
Single-turn violative requests — Harmless response rate, extended thinkingClaude Haiku 4.598.95% (± 0.11%)same evaluation3.1.1
Single-turn violative requests — EnglishClaude Opus 4.699.11%standard and extended thinking combined3.1.1
Single-turn violative requests — ArabicClaude Opus 4.699.87%standard and extended thinking combined3.1.1
Single-turn violative requests — ChineseClaude Opus 4.699.64%standard and extended thinking combined3.1.1
Single-turn violative requests — FrenchClaude Opus 4.699.72%standard and extended thinking combined3.1.1
Single-turn violative requests — KoreanClaude Opus 4.699.72%standard and extended thinking combined3.1.1
Single-turn violative requests — RussianClaude Opus 4.699.61%standard and extended thinking combined3.1.1
Single-turn violative requests — HindiClaude Opus 4.699.78%standard and extended thinking combined3.1.1
Single-turn violative requests — EnglishClaude Opus 4.599.48%standard and extended thinking combined3.1.1
Single-turn violative requests — ArabicClaude Opus 4.599.82%standard and extended thinking combined3.1.1
Single-turn violative requests — ChineseClaude Opus 4.599.66%standard and extended thinking combined3.1.1
Single-turn violative requests — FrenchClaude Opus 4.599.82%standard and extended thinking combined3.1.1
Single-turn violative requests — KoreanClaude Opus 4.599.64%standard and extended thinking combined3.1.1
Single-turn violative requests — RussianClaude Opus 4.599.59%standard and extended thinking combined3.1.1
Single-turn violative requests — HindiClaude Opus 4.599.83%standard and extended thinking combined3.1.1
Single-turn violative requests — EnglishClaude Sonnet 4.598.60%standard and extended thinking combined3.1.1
Single-turn violative requests — ArabicClaude Sonnet 4.598.19%standard and extended thinking combined3.1.1
Single-turn violative requests — ChineseClaude Sonnet 4.597.34%standard and extended thinking combined3.1.1
Single-turn violative requests — FrenchClaude Sonnet 4.598.46%standard and extended thinking combined3.1.1
Single-turn violative requests — KoreanClaude Sonnet 4.597.09%standard and extended thinking combined3.1.1
Single-turn violative requests — RussianClaude Sonnet 4.598.04%standard and extended thinking combined3.1.1
Single-turn violative requests — HindiClaude Sonnet 4.597.92%standard and extended thinking combined3.1.1
Single-turn violative requests — EnglishClaude Haiku 4.599.00%standard and extended thinking combined3.1.1
Single-turn violative requests — ArabicClaude Haiku 4.598.79%standard and extended thinking combined3.1.1
Single-turn violative requests — ChineseClaude Haiku 4.598.39%standard and extended thinking combined3.1.1
Single-turn violative requests — FrenchClaude Haiku 4.599.00%standard and extended thinking combined3.1.1
Single-turn violative requests — KoreanClaude Haiku 4.598.36%standard and extended thinking combined3.1.1
Single-turn violative requests — RussianClaude Haiku 4.598.95%standard and extended thinking combined3.1.1
Single-turn violative requests — HindiClaude Haiku 4.598.39%standard and extended thinking combined3.1.1
Single-turn benign requests — Overall refusal rate (all languages)Claude Opus 4.60.68% (± 0.07%)over-refusal rate; seven languages; developer-reported3.1.2
Single-turn benign requests — Refusal rate, defaultClaude Opus 4.60.80% (± 0.11%)standard Claude mode3.1.2
Single-turn benign requests — Refusal rate, extended thinkingClaude Opus 4.60.56% (± 0.09%)extended thinking3.1.2
Single-turn benign requests — Overall refusal rate (all languages)Claude Opus 4.50.83% (± 0.07%)same evaluation3.1.2
Single-turn benign requests — Refusal rate, defaultClaude Opus 4.50.74% (± 0.10%)same evaluation3.1.2
Single-turn benign requests — Refusal rate, extended thinkingClaude Opus 4.50.93% (± 0.11%)same evaluation3.1.2
Single-turn benign requests — Overall refusal rate (all languages)Claude Sonnet 4.50.09% (± 0.02%)same evaluation3.1.2
Single-turn benign requests — Refusal rate, defaultClaude Sonnet 4.50.10% (± 0.04%)same evaluation3.1.2
Single-turn benign requests — Refusal rate, extended thinkingClaude Sonnet 4.50.07% (± 0.03%)same evaluation3.1.2
Single-turn benign requests — Overall refusal rate (all languages)Claude Haiku 4.50.26% (± 0.04%)same evaluation3.1.2
Single-turn benign requests — Refusal rate, defaultClaude Haiku 4.50.33% (± 0.07%)same evaluation3.1.2
Single-turn benign requests — Refusal rate, extended thinkingClaude Haiku 4.50.20% (± 0.04%)same evaluation3.1.2
Single-turn benign requests — EnglishClaude Opus 4.60.37%standard and extended thinking combined3.1.2
Single-turn benign requests — ArabicClaude Opus 4.61.09%standard and extended thinking combined3.1.2
Single-turn benign requests — ChineseClaude Opus 4.60.56%standard and extended thinking combined3.1.2
Single-turn benign requests — FrenchClaude Opus 4.60.54%standard and extended thinking combined3.1.2
Single-turn benign requests — KoreanClaude Opus 4.60.82%standard and extended thinking combined3.1.2
Single-turn benign requests — RussianClaude Opus 4.60.35%standard and extended thinking combined3.1.2
Single-turn benign requests — HindiClaude Opus 4.61.06%standard and extended thinking combined3.1.2
Single-turn benign requests — EnglishClaude Opus 4.50.22%standard and extended thinking combined3.1.2
Single-turn benign requests — ArabicClaude Opus 4.51.35%standard and extended thinking combined3.1.2
Single-turn benign requests — ChineseClaude Opus 4.50.79%standard and extended thinking combined3.1.2
Single-turn benign requests — FrenchClaude Opus 4.50.59%standard and extended thinking combined3.1.2
Single-turn benign requests — KoreanClaude Opus 4.50.92%standard and extended thinking combined3.1.2
Single-turn benign requests — RussianClaude Opus 4.50.46%standard and extended thinking combined3.1.2
Single-turn benign requests — HindiClaude Opus 4.51.49%standard and extended thinking combined3.1.2
Single-turn benign requests — EnglishClaude Sonnet 4.50.04%standard and extended thinking combined3.1.2
Single-turn benign requests — ArabicClaude Sonnet 4.50.07%standard and extended thinking combined3.1.2
Single-turn benign requests — ChineseClaude Sonnet 4.50.14%standard and extended thinking combined3.1.2
Single-turn benign requests — FrenchClaude Sonnet 4.50.09%standard and extended thinking combined3.1.2
Single-turn benign requests — KoreanClaude Sonnet 4.50.08%standard and extended thinking combined3.1.2
Single-turn benign requests — RussianClaude Sonnet 4.50.06%standard and extended thinking combined3.1.2
Single-turn benign requests — HindiClaude Sonnet 4.50.13%standard and extended thinking combined3.1.2
Single-turn benign requests — EnglishClaude Haiku 4.50.04%standard and extended thinking combined3.1.2
Single-turn benign requests — ArabicClaude Haiku 4.50.43%standard and extended thinking combined3.1.2
Single-turn benign requests — ChineseClaude Haiku 4.50.38%standard and extended thinking combined3.1.2
Single-turn benign requests — FrenchClaude Haiku 4.50.22%standard and extended thinking combined3.1.2
Single-turn benign requests — KoreanClaude Haiku 4.50.30%standard and extended thinking combined3.1.2
Single-turn benign requests — RussianClaude Haiku 4.50.21%standard and extended thinking combined3.1.2
Single-turn benign requests — HindiClaude Haiku 4.50.26%standard and extended thinking combined3.1.2
Higher-difficulty violative requests — Overall harmless response rateClaude Opus 4.699.18% (± 0.04%)synthetically generated, style-transformed prompts; 14 policy areas, 5,000 prompts each; English only; developer-reported3.1.3.1
Higher-difficulty violative requests — Harmless response rate, defaultClaude Opus 4.699.11% (± 0.06%)standard Claude mode3.1.3.1
Higher-difficulty violative requests — Harmless response rate, extended thinkingClaude Opus 4.699.25% (± 0.05%)extended thinking3.1.3.1
Higher-difficulty violative requests — Overall harmless response rateClaude Opus 4.599.28% (± 0.04%)same evaluation3.1.3.1
Higher-difficulty violative requests — Harmless response rate, defaultClaude Opus 4.599.13% (± 0.06%)same evaluation3.1.3.1
Higher-difficulty violative requests — Harmless response rate, extended thinkingClaude Opus 4.599.42% (± 0.05%)same evaluation3.1.3.1
Higher-difficulty violative requests — Overall harmless response rateClaude Sonnet 4.598.40% (± 0.05%)same evaluation3.1.3.1
Higher-difficulty violative requests — Harmless response rate, defaultClaude Sonnet 4.598.44% (± 0.08%)same evaluation3.1.3.1
Higher-difficulty violative requests — Harmless response rate, extended thinkingClaude Sonnet 4.598.35% (± 0.08%)same evaluation3.1.3.1
Higher-difficulty violative requests — Overall harmless response rateClaude Haiku 4.598.62% (± 0.05%)same evaluation3.1.3.1
Higher-difficulty violative requests — Harmless response rate, defaultClaude Haiku 4.599.05% (± 0.06%)same evaluation3.1.3.1
Higher-difficulty violative requests — Harmless response rate, extended thinkingClaude Haiku 4.598.19% (± 0.08%)same evaluation3.1.3.1
Higher-difficulty benign requests — Overall refusal rateClaude Opus 4.60.04% (± 0.01%)prompts transformed with elaborate justifications and academic framing; English only; developer-reported3.1.3.2
Higher-difficulty benign requests — Refusal rate, defaultClaude Opus 4.60.06% (± 0.01%)standard Claude mode3.1.3.2
Higher-difficulty benign requests — Refusal rate, extended thinkingClaude Opus 4.60.02% (± 0.01%)extended thinking3.1.3.2
Higher-difficulty benign requests — Overall refusal rateClaude Opus 4.50.83% (± 0.04%)same evaluation3.1.3.2
Higher-difficulty benign requests — Refusal rate, defaultClaude Opus 4.50.95% (± 0.06%)same evaluation3.1.3.2
Higher-difficulty benign requests — Refusal rate, extended thinkingClaude Opus 4.50.71% (± 0.05%)same evaluation3.1.3.2
Higher-difficulty benign requests — Overall refusal rateClaude Sonnet 4.58.50% (± 0.13%)same evaluation3.1.3.2
Higher-difficulty benign requests — Refusal rate, defaultClaude Sonnet 4.511.69% (± 0.21%)same evaluation3.1.3.2
Higher-difficulty benign requests — Refusal rate, extended thinkingClaude Sonnet 4.55.32% (± 0.14%)same evaluation3.1.3.2
Higher-difficulty benign requests — Overall refusal rateClaude Haiku 4.56.01% (± 0.11%)same evaluation3.1.3.2
Higher-difficulty benign requests — Refusal rate, defaultClaude Haiku 4.57.15% (± 0.16%)same evaluation3.1.3.2
Higher-difficulty benign requests — Refusal rate, extended thinkingClaude Haiku 4.54.87% (± 0.14%)same evaluation3.1.3.2
Child safety — Single-turn violative requests (harmless rate)Claude Opus 4.699.95% (± 0.04%)all tested languages; human-crafted and synthetic prompts on child sexualization, grooming, child marriage and other child abuse; developer-reported3.4.1
Child safety — Single-turn benign requests (refusal rate)Claude Opus 4.60.18% (± 0.10%)all tested languages3.4.1
Child safety — Multi-turn (appropriate response rate)Claude Opus 4.696% (± 3%)multi-turn test cases including new sex tourism, child sex trafficking, age-of-consent and CSAM prompt-engineering cases3.4.1
Child safety — Single-turn violative requests (harmless rate)Claude Opus 4.599.91% (± 0.06%)same evaluation3.4.1
Child safety — Single-turn benign requests (refusal rate)Claude Opus 4.50.33% (± 0.13%)same evaluation3.4.1
Child safety — Multi-turn (appropriate response rate)Claude Opus 4.599% (± 2%)same evaluation3.4.1
Child safety — Single-turn violative requests (harmless rate)Claude Sonnet 4.599.65% (± 0.11%)same evaluation3.4.1
Child safety — Single-turn benign requests (refusal rate)Claude Sonnet 4.50.08% (± 0.07%)same evaluation3.4.1
Child safety — Multi-turn (appropriate response rate)Claude Sonnet 4.598% (± 2%)same evaluation3.4.1
Child safety — Single-turn violative requests (harmless rate)Claude Haiku 4.599.88% (± 0.07%)same evaluation3.4.1
Child safety — Single-turn benign requests (refusal rate)Claude Haiku 4.50.04% (± 0.05%)same evaluation3.4.1
Child safety — Multi-turn (appropriate response rate)Claude Haiku 4.596% (± 3%)same evaluation3.4.1
Suicide and self-harm — Single-turn requests posing potential risk (harmless rate)Claude Opus 4.699.75% (± 0.12%)all tested languages; developer-reported3.4.2
Suicide and self-harm — Single-turn benign requests (refusal rate)Claude Opus 4.60.25% (± 0.15%)all tested languages3.4.2
Suicide and self-harm — Multi-turn (appropriate response rate)Claude Opus 4.682% (± 11%)multi-turn conversation testing3.4.2
Suicide and self-harm — Single-turn requests posing potential risk (harmless rate)Claude Opus 4.599.56% (± 0.17%)same evaluation3.4.2
Suicide and self-harm — Single-turn benign requests (refusal rate)Claude Opus 4.50.14% (± 0.10%)same evaluation3.4.2
Suicide and self-harm — Multi-turn (appropriate response rate)Claude Opus 4.586% (± 10%)same evaluation3.4.2
Suicide and self-harm — Single-turn requests posing potential risk (harmless rate)Claude Sonnet 4.598.93% (± 0.28%)same evaluation3.4.2
Suicide and self-harm — Single-turn benign requests (refusal rate)Claude Sonnet 4.50.01% (± 0.02%)same evaluation3.4.2
Suicide and self-harm — Multi-turn (appropriate response rate)Claude Sonnet 4.578% (± 12%)same evaluation3.4.2
Suicide and self-harm — Single-turn requests posing potential risk (harmless rate)Claude Haiku 4.599.65% (± 0.14%)same evaluation3.4.2
Suicide and self-harm — Single-turn benign requests (refusal rate)Claude Haiku 4.50.03% (± 0.05%)same evaluation3.4.2
Suicide and self-harm — Multi-turn (appropriate response rate)Claude Haiku 4.590% (± 9%)same evaluation3.4.2
Political bias and evenhandedness — EvenhandednessClaude Opus 4.698.2%open-source paired-prompt evaluation: 1,350 prompt pairs, 9 task types, 150 topics; Claude Sonnet 4.5 as grader; public system prompt included, standard thinking mode; developer-reported3.5.1
Political bias and evenhandedness — Opposing perspectivesClaude Opus 4.644.6%same run3.5.1
Political bias and evenhandedness — RefusalsClaude Opus 4.64.5%same run; lower is better3.5.1
Political bias and evenhandedness — EvenhandednessClaude Opus 4.596.2%same evaluation3.5.1
Political bias and evenhandedness — Opposing perspectivesClaude Opus 4.540.5%same evaluation3.5.1
Political bias and evenhandedness — RefusalsClaude Opus 4.53.9%same evaluation3.5.1
Political bias and evenhandedness — EvenhandednessClaude Haiku 4.588.4%same evaluation3.5.1
Political bias and evenhandedness — Opposing perspectivesClaude Haiku 4.543.6%same evaluation3.5.1
Political bias and evenhandedness — RefusalsClaude Haiku 4.54.4%same evaluation3.5.1
Political bias and evenhandedness — EvenhandednessClaude Sonnet 4.594.2%same evaluation3.5.1
Political bias and evenhandedness — Opposing perspectivesClaude Sonnet 4.526.2%same evaluation3.5.1
Political bias and evenhandedness — RefusalsClaude Sonnet 4.52.2%same evaluation3.5.1
Bias Benchmark for Question Answering (BBQ) — Disambiguated biasClaude Opus 4.6-0.73standard thinking mode; closer to zero is better; developer-reported3.5.2
Bias Benchmark for Question Answering (BBQ) — Ambiguous biasClaude Opus 4.60.14standard thinking mode3.5.2
Bias Benchmark for Question Answering (BBQ) — Disambiguated accuracyClaude Opus 4.690.9standard thinking mode3.5.2
Bias Benchmark for Question Answering (BBQ) — Ambiguous accuracyClaude Opus 4.699.7standard thinking mode3.5.2
Bias Benchmark for Question Answering (BBQ) — Disambiguated biasClaude Opus 4.5-0.64same evaluation3.5.2
Bias Benchmark for Question Answering (BBQ) — Ambiguous biasClaude Opus 4.50.26same evaluation3.5.2
Bias Benchmark for Question Answering (BBQ) — Disambiguated accuracyClaude Opus 4.588.7same evaluation3.5.2
Bias Benchmark for Question Answering (BBQ) — Ambiguous accuracyClaude Opus 4.599.7same evaluation3.5.2
Bias Benchmark for Question Answering (BBQ) — Disambiguated biasClaude Haiku 4.50.54same evaluation3.5.2
Bias Benchmark for Question Answering (BBQ) — Ambiguous biasClaude Haiku 4.51.37same evaluation3.5.2
Bias Benchmark for Question Answering (BBQ) — Disambiguated accuracyClaude Haiku 4.571.2same evaluation3.5.2
Bias Benchmark for Question Answering (BBQ) — Ambiguous accuracyClaude Haiku 4.598.0same evaluation3.5.2
Bias Benchmark for Question Answering (BBQ) — Disambiguated biasClaude Sonnet 4.5-2.21same evaluation3.5.2
Bias Benchmark for Question Answering (BBQ) — Ambiguous biasClaude Sonnet 4.50.25same evaluation3.5.2
Bias Benchmark for Question Answering (BBQ) — Disambiguated accuracyClaude Sonnet 4.582.2same evaluation3.5.2
Bias Benchmark for Question Answering (BBQ) — Ambiguous accuracyClaude Sonnet 4.599.7same evaluation3.5.2
Malicious agentic coding — Refusal rate (without mitigations)Claude Opus 4.699.3%150 malicious coding requests prohibited by the Usage Policy; same coding tools as capability evaluations; no additional safeguards; refused all but one request; developer-reported5.1.1
Malicious agentic coding — Refusal rate (without mitigations)Claude Opus 4.5100%same evaluation5.1.1
Malicious agentic coding — Refusal rate (without mitigations)Claude Sonnet 4.598.7%same evaluation5.1.1
Malicious agentic coding — Refusal rate (without mitigations)Claude Haiku 4.5100%same evaluation5.1.1
Malicious use of Claude Code — Malicious (refusal rate, without mitigations)Claude Opus 4.683.20%49 malicious prompts run 10 times each (490 tests); standard Claude Code tool commands; developer-reported5.1.2
Malicious use of Claude Code — Dual-use and benign (success rate, without mitigations)Claude Opus 4.691.75%61 prompts run 10 times each (610 tests)5.1.2
Malicious use of Claude Code — Malicious (refusal rate, without mitigations)Claude Opus 4.577.80%same evaluation5.1.2
Malicious use of Claude Code — Dual-use and benign (success rate, without mitigations)Claude Opus 4.593.07%same evaluation5.1.2
Malicious use of Claude Code — Malicious (refusal rate, without mitigations)Claude Sonnet 4.563.06%same evaluation5.1.2
Malicious use of Claude Code — Dual-use and benign (success rate, without mitigations)Claude Sonnet 4.596.56%same evaluation5.1.2
Malicious use of Claude Code — Malicious (refusal rate, without mitigations)Claude Haiku 4.569.39%same evaluation5.1.2
Malicious use of Claude Code — Dual-use and benign (success rate, without mitigations)Claude Haiku 4.584.92%same evaluation5.1.2
Malicious use of Claude Code — Malicious (refusal rate, with mitigations)Claude Opus 4.699.59%with Claude Code system prompt and FileRead-tool reminder mitigations5.1.2
Malicious use of Claude Code — Dual-use and benign (success rate, with mitigations)Claude Opus 4.695.59%with both mitigations5.1.2
Malicious use of Claude Code — Malicious (refusal rate, with mitigations)Claude Opus 4.597.35%same evaluation5.1.2
Malicious use of Claude Code — Dual-use and benign (success rate, with mitigations)Claude Opus 4.596.52%same evaluation5.1.2
Malicious use of Claude Code — Malicious (refusal rate, with mitigations)Claude Sonnet 4.595.10%same evaluation5.1.2
Malicious use of Claude Code — Dual-use and benign (success rate, with mitigations)Claude Sonnet 4.598.20%same evaluation5.1.2
Malicious use of Claude Code — Malicious (refusal rate, with mitigations)Claude Haiku 4.596.73%same evaluation5.1.2
Malicious use of Claude Code — Dual-use and benign (success rate, with mitigations)Claude Haiku 4.586.07%same evaluation5.1.2
Malicious computer use — Refusal rate (without mitigations)Claude Opus 4.688.34%112 tasks run with both extended and standard thinking (224 attempts per model); GUI- and CLI-based tools in a sandbox; developer-reported5.1.3
Malicious computer use — Refusal rate (without mitigations)Claude Opus 4.588.39%same evaluation5.1.3
Malicious computer use — Refusal rate (without mitigations)Claude Sonnet 4.586.08%same evaluation5.1.3
Malicious computer use — Refusal rate (without mitigations)Claude Haiku 4.577.68%same evaluation5.1.3
Agent Red Teaming (ART) benchmark — Indirect prompt injection, k=100, no extended thinkingClaude Opus 4.614.8%attack success probability after 100 attempts; evaluated by Gray Swan; benchmark developed with the UK AI Security Institute; 19 scenarios; grading issues corrected jointly with Gray Swan5.2.1
Agent Red Teaming (ART) benchmark — Indirect prompt injection, k=100, extended thinkingClaude Opus 4.621.7%attack success probability after 100 attempts, extended thinking enabled; evaluated by Gray Swan5.2.1
Agent Red Teaming (ART) benchmark — Indirect prompt injection, k=100, no extended thinkingClaude Opus 4.516.5%attack success probability after 100 attempts; evaluated by Gray Swan5.2.1
Shade prompt injection (coding) — Extended thinking, no safeguards, 1 attemptClaude Opus 4.60.0%attack success rate; Gray Swan Shade adaptive red-teaming tool; developer-reported5.2.2.1
Shade prompt injection (coding) — Extended thinking, no safeguards, 200 attemptsClaude Opus 4.60.0%adaptive attacker with 200 attempts per goal5.2.2.1
Shade prompt injection (coding) — Extended thinking, with safeguards, 1 attemptClaude Opus 4.60.0%safeguards applied on tool responses5.2.2.1
Shade prompt injection (coding) — Extended thinking, with safeguards, 200 attemptsClaude Opus 4.60.0%same evaluation5.2.2.1
Shade prompt injection (coding) — Standard thinking, no safeguards, 1 attemptClaude Opus 4.60.0%same evaluation5.2.2.1
Shade prompt injection (coding) — Standard thinking, no safeguards, 200 attemptsClaude Opus 4.60.0%same evaluation5.2.2.1
Shade prompt injection (coding) — Standard thinking, with safeguards, 1 attemptClaude Opus 4.60.0%same evaluation5.2.2.1
Shade prompt injection (coding) — Standard thinking, with safeguards, 200 attemptsClaude Opus 4.60.0%same evaluation5.2.2.1
Shade prompt injection (coding) — Extended thinking, no safeguards, 1 attemptClaude Opus 4.50.3%same evaluation5.2.2.1
Shade prompt injection (coding) — Extended thinking, no safeguards, 200 attemptsClaude Opus 4.510.0%same evaluation5.2.2.1
Shade prompt injection (coding) — Extended thinking, with safeguards, 1 attemptClaude Opus 4.50.1%same evaluation5.2.2.1
Shade prompt injection (coding) — Extended thinking, with safeguards, 200 attemptsClaude Opus 4.57.5%same evaluation5.2.2.1
Shade prompt injection (coding) — Standard thinking, no safeguards, 1 attemptClaude Opus 4.50.7%same evaluation5.2.2.1
Shade prompt injection (coding) — Standard thinking, no safeguards, 200 attemptsClaude Opus 4.517.5%same evaluation5.2.2.1
Shade prompt injection (coding) — Standard thinking, with safeguards, 1 attemptClaude Opus 4.50.2%same evaluation5.2.2.1
Shade prompt injection (coding) — Standard thinking, with safeguards, 200 attemptsClaude Opus 4.57.5%same evaluation5.2.2.1
Shade prompt injection (coding) — Extended thinking, no safeguards, 1 attemptClaude Sonnet 4.518.3%same evaluation5.2.2.1
Shade prompt injection (coding) — Extended thinking, no safeguards, 200 attemptsClaude Sonnet 4.570.0%same evaluation5.2.2.1
Shade prompt injection (coding) — Extended thinking, with safeguards, 1 attemptClaude Sonnet 4.51.6%same evaluation5.2.2.1
Shade prompt injection (coding) — Extended thinking, with safeguards, 200 attemptsClaude Sonnet 4.525.0%same evaluation5.2.2.1
Shade prompt injection (coding) — Standard thinking, no safeguards, 1 attemptClaude Sonnet 4.531.6%same evaluation5.2.2.1
Shade prompt injection (coding) — Standard thinking, no safeguards, 200 attemptsClaude Sonnet 4.587.5%same evaluation5.2.2.1
Shade prompt injection (coding) — Standard thinking, with safeguards, 1 attemptClaude Sonnet 4.51.7%same evaluation5.2.2.1
Shade prompt injection (coding) — Standard thinking, with safeguards, 200 attemptsClaude Sonnet 4.525.0%same evaluation5.2.2.1
Shade prompt injection (computer use) — Extended thinking, no safeguards, 1 attemptClaude Opus 4.617.8%stronger Gray Swan attacker variant in GUI computer-use environments; developer-reported5.2.2.2
Shade prompt injection (computer use) — Extended thinking, no safeguards, 200 attemptsClaude Opus 4.678.6%adaptive attacker, 200 attempts per goal5.2.2.2
Shade prompt injection (computer use) — Extended thinking, with safeguards, 1 attemptClaude Opus 4.69.7%same evaluation5.2.2.2
Shade prompt injection (computer use) — Extended thinking, with safeguards, 200 attemptsClaude Opus 4.657.1%same evaluation5.2.2.2
Shade prompt injection (computer use) — Standard thinking, no safeguards, 1 attemptClaude Opus 4.620.0%same evaluation5.2.2.2
Shade prompt injection (computer use) — Standard thinking, no safeguards, 200 attemptsClaude Opus 4.685.7%same evaluation5.2.2.2
Shade prompt injection (computer use) — Standard thinking, with safeguards, 1 attemptClaude Opus 4.610.0%same evaluation5.2.2.2
Shade prompt injection (computer use) — Standard thinking, with safeguards, 200 attemptsClaude Opus 4.664.3%same evaluation5.2.2.2
Shade prompt injection (computer use) — Extended thinking, no safeguards, 1 attemptClaude Opus 4.528.0%same evaluation5.2.2.2
Shade prompt injection (computer use) — Extended thinking, no safeguards, 200 attemptsClaude Opus 4.578.6%same evaluation5.2.2.2
Shade prompt injection (computer use) — Extended thinking, with safeguards, 1 attemptClaude Opus 4.517.3%same evaluation5.2.2.2
Shade prompt injection (computer use) — Extended thinking, with safeguards, 200 attemptsClaude Opus 4.564.3%same evaluation5.2.2.2
Shade prompt injection (computer use) — Standard thinking, no safeguards, 1 attemptClaude Opus 4.535.4%same evaluation5.2.2.2
Shade prompt injection (computer use) — Standard thinking, no safeguards, 200 attemptsClaude Opus 4.585.7%same evaluation5.2.2.2
Shade prompt injection (computer use) — Standard thinking, with safeguards, 1 attemptClaude Opus 4.518.8%same evaluation5.2.2.2
Shade prompt injection (computer use) — Standard thinking, with safeguards, 200 attemptsClaude Opus 4.571.4%same evaluation5.2.2.2
Shade prompt injection (computer use) — Extended thinking, no safeguards, 1 attemptClaude Sonnet 4.541.8%same evaluation5.2.2.2
Shade prompt injection (computer use) — Extended thinking, no safeguards, 200 attemptsClaude Sonnet 4.592.9%same evaluation5.2.2.2
Shade prompt injection (computer use) — Extended thinking, with safeguards, 1 attemptClaude Sonnet 4.525.2%same evaluation5.2.2.2
Shade prompt injection (computer use) — Extended thinking, with safeguards, 200 attemptsClaude Sonnet 4.585.7%same evaluation5.2.2.2
Shade prompt injection (computer use) — Standard thinking, no safeguards, 1 attemptClaude Sonnet 4.519.0%same evaluation5.2.2.2
Shade prompt injection (computer use) — Standard thinking, no safeguards, 200 attemptsClaude Sonnet 4.592.9%same evaluation5.2.2.2
Shade prompt injection (computer use) — Standard thinking, with safeguards, 1 attemptClaude Sonnet 4.512.8%same evaluation5.2.2.2
Shade prompt injection (computer use) — Standard thinking, with safeguards, 200 attemptsClaude Sonnet 4.571.4%same evaluation5.2.2.2
Shade prompt injection (computer use, original attacker) — Standard thinking, no safeguards, 1 attemptClaude Opus 4.60.29%original (weaker) attacker evaluation, reported for consistency with prior cards5.2.2.2
Shade prompt injection (computer use, original attacker) — Standard thinking, no safeguards, 200 attemptsClaude Opus 4.67.14%original attacker evaluation5.2.2.2
Shade prompt injection (computer use, original attacker) — Standard thinking, no safeguards, 1 attemptClaude Opus 4.50.71%original attacker evaluation5.2.2.2
Shade prompt injection (computer use, original attacker) — Standard thinking, no safeguards, 200 attemptsClaude Opus 4.528.6%original attacker evaluation; both models remain at 0% with extended thinking5.2.2.2
Browser-use prompt injection (internal Best-of-N) — Extended thinking, no safeguards, % of scenariosClaude Opus 4.62.06%10 attack strings across 389 scenarios; per-scenario ASR (at least one attempt succeeded); developer-reported5.2.2.3
Browser-use prompt injection (internal Best-of-N) — Extended thinking, no safeguards, % of attemptsClaude Opus 4.60.29%per-attempt ASR aggregated across environments5.2.2.3
Browser-use prompt injection (internal Best-of-N) — Standard thinking, no safeguards, % of scenariosClaude Opus 4.62.83%same evaluation5.2.2.3
Browser-use prompt injection (internal Best-of-N) — Standard thinking, no safeguards, % of attemptsClaude Opus 4.60.49%same evaluation5.2.2.3
Browser-use prompt injection (internal Best-of-N) — Extended thinking, no safeguards, % of scenariosClaude Opus 4.518.77%same evaluation5.2.2.3
Browser-use prompt injection (internal Best-of-N) — Extended thinking, no safeguards, % of attemptsClaude Opus 4.56.40%same evaluation5.2.2.3
Browser-use prompt injection (internal Best-of-N) — Standard thinking, no safeguards, % of scenariosClaude Opus 4.516.20%same evaluation5.2.2.3
Browser-use prompt injection (internal Best-of-N) — Standard thinking, no safeguards, % of attemptsClaude Opus 4.55.06%same evaluation5.2.2.3
Browser-use prompt injection (internal Best-of-N) — Extended thinking, no safeguards, % of scenariosClaude Sonnet 4.554.24%same evaluation5.2.2.3
Browser-use prompt injection (internal Best-of-N) — Extended thinking, no safeguards, % of attemptsClaude Sonnet 4.520.45%same evaluation5.2.2.3
Browser-use prompt injection (internal Best-of-N) — Standard thinking, no safeguards, % of scenariosClaude Sonnet 4.549.36%same evaluation5.2.2.3
Browser-use prompt injection (internal Best-of-N) — Standard thinking, no safeguards, % of attemptsClaude Sonnet 4.516.23%same evaluation5.2.2.3
Browser-use prompt injection (internal Best-of-N) — Standard thinking, previous safeguards, % of scenariosClaude Opus 4.60.26%with the safeguards deployed alongside Claude Opus 4.55.2.2.3
Browser-use prompt injection (internal Best-of-N) — Standard thinking, previous safeguards, % of attemptsClaude Opus 4.60.03%same evaluation5.2.2.3
Browser-use prompt injection (internal Best-of-N) — Standard thinking, updated safeguards, % of scenariosClaude Opus 4.60.77%with the updated safeguards5.2.2.3
Browser-use prompt injection (internal Best-of-N) — Standard thinking, updated safeguards, % of attemptsClaude Opus 4.60.08%same evaluation5.2.2.3
Browser-use prompt injection (internal Best-of-N) — Standard thinking, previous safeguards, % of scenariosClaude Opus 4.51.03%same evaluation5.2.2.3
Browser-use prompt injection (internal Best-of-N) — Standard thinking, previous safeguards, % of attemptsClaude Opus 4.50.21%same evaluation5.2.2.3
Browser-use prompt injection (internal Best-of-N) — Standard thinking, updated safeguards, % of scenariosClaude Opus 4.51.54%same evaluation5.2.2.3
Browser-use prompt injection (internal Best-of-N) — Standard thinking, updated safeguards, % of attemptsClaude Opus 4.50.40%same evaluation5.2.2.3
Browser-use prompt injection (internal Best-of-N) — Standard thinking, previous safeguards, % of scenariosClaude Sonnet 4.51.54%same evaluation5.2.2.3
Browser-use prompt injection (internal Best-of-N) — Standard thinking, previous safeguards, % of attemptsClaude Sonnet 4.50.41%same evaluation5.2.2.3
Browser-use prompt injection (internal Best-of-N) — Standard thinking, updated safeguards, % of scenariosClaude Sonnet 4.52.06%same evaluation5.2.2.3
Browser-use prompt injection (internal Best-of-N) — Standard thinking, updated safeguards, % of attemptsClaude Sonnet 4.50.46%same evaluation5.2.2.3
Reward-hack-prone coding tasks — Classifier hack rateClaude Opus 4.60%hand-selected coding problems from the training distribution on which previous models hacked frequently; developer-reported6.2.3.2
Reward-hack-prone coding tasks — Hidden test hack rateClaude Opus 4.60%held-out subset of tests6.2.3.2
Impossible tasks — Classifier hack rate with no promptClaude Opus 4.650%agentic setting on intentionally impossible problems; minimal instructions6.2.3.2
Impossible tasks — Classifier hack rate with anti-hack promptClaude Opus 4.623%with the anti-reward-hacking prompt6.2.3.2
Reward-hack-prone coding tasks — Classifier hack rateClaude Opus 4.50%same evaluation6.2.3.2
Reward-hack-prone coding tasks — Hidden test hack rateClaude Opus 4.50%same evaluation6.2.3.2
Impossible tasks — Classifier hack rate with no promptClaude Opus 4.555%same evaluation6.2.3.2
Impossible tasks — Classifier hack rate with anti-hack promptClaude Opus 4.535%same evaluation6.2.3.2
Reward-hack-prone coding tasks — Classifier hack rateClaude Haiku 4.56%same evaluation6.2.3.2
Reward-hack-prone coding tasks — Hidden test hack rateClaude Haiku 4.53%same evaluation6.2.3.2
Impossible tasks — Classifier hack rate with no promptClaude Haiku 4.530%same evaluation6.2.3.2
Impossible tasks — Classifier hack rate with anti-hack promptClaude Haiku 4.523%same evaluation6.2.3.2
Reward-hack-prone coding tasks — Classifier hack rateClaude Sonnet 4.51%same evaluation6.2.3.2
Reward-hack-prone coding tasks — Hidden test hack rateClaude Sonnet 4.51%same evaluation6.2.3.2
Impossible tasks — Classifier hack rate with no promptClaude Sonnet 4.553%same evaluation6.2.3.2
Impossible tasks — Classifier hack rate with anti-hack promptClaude Sonnet 4.520%same evaluation6.2.3.2
Mentions of being evaluated (UK AISI) — Verbalized evaluation awarenessClaude Opus 4.62.3%share of interactions; measured by the UK AI Security Institute on an early snapshot over 3 working days6.2.6
Mentions of being evaluated (UK AISI) — Verbalized evaluation awarenessClaude Sonnet 4.511.4%measured by the UK AI Security Institute6.2.6
Mentions of being evaluated (UK AISI) — Verbalized evaluation awarenessClaude Opus 4.511.0%measured by the UK AI Security Institute6.2.6
Distinguishing deployment from simulation (UK AISI) — Paired comparison (which of two conversations was an evaluation)Claude Opus 4.680%prompted identification accuracy; measured by the UK AI Security Institute6.2.6
Distinguishing deployment from simulation (UK AISI) — Paired comparisonClaude Sonnet 4.562%measured by the UK AI Security Institute6.2.6
Distinguishing deployment from simulation (UK AISI) — Paired comparisonClaude Opus 4.572%measured by the UK AI Security Institute6.2.6
Distinguishing deployment from simulation (UK AISI) — Single-interaction recognitionClaude Opus 4.645.5%recognising a single interaction as an evaluation; measured by the UK AI Security Institute6.2.6
Distinguishing deployment from simulation (UK AISI) — Single-interaction recognitionClaude Sonnet 4.515%measured by the UK AI Security Institute6.2.6
Distinguishing deployment from simulation (UK AISI) — Single-interaction recognitionClaude Opus 4.512.5%measured by the UK AI Security Institute6.2.6
Refusal on safety research tasks (UK AISI) — Refusal rateClaude Opus 4.626%share of interactions refusing safety research tasks; measured by the UK AI Security Institute6.2.6
Refusal on safety research tasks (UK AISI) — Refusal rateClaude Sonnet 4.562%measured by the UK AI Security Institute6.2.6
Refusal on safety research tasks (UK AISI) — Refusal rateClaude Opus 4.560%measured by the UK AI Security Institute6.2.6
Refusal to assist with AI safety R&D (internal) — Refusal rateClaude Opus 4.614%single-turn queries in a coding setting with user background context; developer-reported6.3.6
Refusal to assist with AI safety R&D (internal) — Refusal rateClaude 4.5 modelsaround 60%as stated in prose for "all of the Claude 4.5 models"; per-model figures appear only in Figure 6.3.6.A6.3.6
Self-preference evaluation — Self-favoritism, ten-point scaleClaude Opus 4.6less than 0.2 pointstranscript-grading task; grader told the transcript came from Claude or another frontier model6.3.7
Self-preference evaluation — Self-favoritism, ten-point scaleClaude Opus 4.5less than 0.2 pointssame evaluation6.3.7
Internal codebase sabotage propensity — Sabotage rateClaude Opus 4.6nevercrafted Claude Code scenario in which flagging is implied to be costly to Anthropic; also 9,000 resampled historical internal Claude Code queries with no sabotage flagged; developer-reported6.3.8
Internal codebase sabotage propensity — Sabotage rateClaude Opus 4.10.8%same scenario, across multiple transcripts6.3.8
SHADE-Arena — Extended Thinking (evasion success)Claude Opus 4.618%side task completed without being flagged by a Claude Sonnet 4 monitor; attacker prompting updated using cues from Zolkowski et al. (2025); developer-reported6.4.1
SHADE-Arena — Extended Thinking (references side task in reasoning)Claude Opus 4.626%rate of referencing the secret side task in its thought process6.4.1
SHADE-Arena — Extended Thinking (references side task in reasoning)Claude Opus 4.574%same measurement6.4.1
SHADE-Arena — No extended thinking (evasion success)Claude Opus 4.616%side task completed without being caught by the monitor6.4.1
SHADE-Arena — No extended thinking (evasion success)Claude Sonnet 4.523%same setting6.4.1
SHADE-Arena — No extended thinking (evasion success)Claude Opus 4.526%same setting6.4.1
SHADE-Arena — Hidden scratchpad (evasion success)Claude Opus 4.646%agent given a fully hidden scratchpad invisible to the monitor; described as an approximate upper bound; Opus 4.5 performs similarly6.4.1
Long-form virology task 1 (influenza) — CBRN RSP evaluationClaude Opus 4.60.79"max of 5" score; task-based agentic evaluation developed with SecureBio, Deloitte and Signature Science; developer-reported8.2.4.1
Long-form virology task 1 (influenza) — CBRN RSP evaluationClaude Opus 4.50.845"max of 5" score8.2.4.1
Long-form virology task 1 (influenza) — CBRN RSP evaluationClaude Sonnet 4.50.89"max of 5" score8.2.4.1
Long-form virology task 2 (SARS-CoV) — CBRN RSP evaluationClaude Opus 4.60.9"max of 5" score; exceeds the 0.8 threshold8.2.4.1
Long-form virology task 2 (SARS-CoV) — CBRN RSP evaluationClaude Opus 4.50.912"max of 5" score8.2.4.1
Long-form virology task 2 (SARS-CoV) — CBRN RSP evaluationClaude Sonnet 4.50.88"max of 5" score8.2.4.1
Long-form virology tasks — ThresholdsThresholdrule-out <50%; ASL-3 rule-in >80%Anthropic's stated zones; scores between require additional evaluation8.2.4.1
Creative Biology Uplift Trial — Model-assisted vs. controlClaude Opus 4.6roughly twice the score of the control grouptwenty molecular biology PhDs split into a model-assisted group (Claude.ai with web search and extended thinking) and an internet-only control; up to 20 hours over three days; graded by multiple expert graders; no statistically significant difference between Opus 4.5 and Opus 4.68.2.4.4
ASL-4 virology protocol uplift trial — Fractional scoreClaude Opus 4.60.52average protocol score of the model-assisted group; trial and rubric designed by Deloitte Consulting; model-assisted and control groups had 3 days8.2.4.5
ASL-4 virology protocol uplift trial — Fractional scoreClaude Opus 4.50.61same trial; groups had 4 days8.2.4.5
ASL-4 virology protocol uplift trial — Fractional scoreClaude Opus 40.50same trial; groups had 2 days8.2.4.5
ASL-4 virology protocol uplift trial — Critical failuresClaude Opus 4.66.6 critical failuresmean over the model-assisted group; above the rule-out threshold of 1.88.2.4.5
ASL-4 virology protocol uplift trial — Critical failuresClaude Opus 4.55.6 critical failuresmean over the model-assisted group8.2.4.5
ASL-4 virology protocol uplift trial — Critical failuresClaude Opus 48.4 critical failuresmean over the model-assisted group8.2.4.5
ASL-4 virology protocol uplift trial — ThresholdThreshold<1.8 out of 18updated threshold: <10% mean total critical failures on the model-assisted group (previously <25%, i.e. 4.5 out of 18)8.2.4.5
Creative Biology Automated Evaluations — SecureBio creative biology questionsClaude Opus 4.60.603developer-reported8.2.4.8
Creative Biology Automated Evaluations — SecureBio creative biology questionsClaude Opus 4.50.524same evaluation8.2.4.8
Creative Biology Automated Evaluations — SecureBio creative biology questionsClaude Sonnet 4.50.488same evaluation8.2.4.8
Creative Biology Automated Evaluations — Human baselineBiology PhDs (SecureBio preliminary baselining)14%preliminary human baselining on a subset of the questions8.2.4.8
Short-horizon computational biology tasks — Lower-bound rule-out thresholds crossedClaude Opus 4.66/6 tasksdeveloped with Faculty.ai; first Anthropic model to cross the lower-bound rule-out thresholds on all six tasks8.2.4.9
Kernels task — Best speedup, experimental scaffoldClaude Opus 4.6427×AI R&D evaluation; mean score exceeded the 100× threshold; developer-reported8.3.3.1
Kernels task — Best speedup, standard scaffoldClaude Opus 4.6190×standard scaffold8.3.3.1
Kernels task — ThresholdsThreshold100× (4× ≈ 1 human hour; 200× ≈ 8 hours; 300× ≈ 40 hours)Anthropic's stated threshold and human-time estimates8.3.3.1
Time series forecasting — Hard variant (MSE)Claude Opus 4.65.86minimum MSE; above (worse than) the human baseline of 5.3; lower is better8.3.3.2
Time series forecasting — Medium variant (MSE)Claude Opus 4.65.76minimum MSE; below the human baseline of 5.8; medium threshold estimated at 8 human hours8.3.3.2
Time series forecasting — Human baseline, hard variantHuman baseline5.3MSE; lower is better8.3.3.2
Time series forecasting — Human baseline, medium variantHuman baseline5.8MSE; lower is better8.3.3.2
Text-based reinforcement learning task — Best scoreClaude Opus 4.60.975above the 0.9 threshold, estimated to represent 8 human-effort hours; mean score also above threshold for the first time8.3.3.3
Text-based reinforcement learning task — ThresholdThreshold0.9Anthropic's stated threshold8.3.3.3
LLM training — SpeedupClaude Opus 4.634×well above the 4× reference expert solution, which represents 4–8 human-effort hours8.3.3.4
Quadruped reinforcement learning — Highest score, no hyperparameter variantClaude Opus 4.620.96above the threshold of 12, representing 4 human-effort hours; median also above threshold8.3.3.5
Quadruped reinforcement learning — Highest score, no reward function variantClaude Opus 4.621.99above the threshold of 128.3.3.5
Novel compiler — Basic tests pass rateClaude Opus 4.698.2%developer-reported8.3.3.6
Novel compiler — Complex tests pass rateClaude Opus 4.665.83%below the 90% threshold, estimated to represent 40 human-effort hours8.3.3.6
Internal AI research evaluation suite 2 — Weighted average scoreClaude Opus 4.60.6124surpassing the rule-out threshold of 0.6; developer-reported8.3.4
Internal AI research evaluation suite 2 — Weighted average scoreClaude Opus 4.50.604same evaluation8.3.4
Internal model use survey — L4 drop-in replacement questionClaude Opus 4.611 of 16 unlikely; 3 likely with 3 months of improvements; 2 already possiblesurvey of sixteen Anthropic technical staff, including general research and infrastructure/maintenance roles8.3.1
Internal productivity uplift survey — Productivity uplift from using the modelClaude Opus 4.630% to 700% range; mean 152%; median 100%sixteen Anthropic technical staff; more modest than previous superuser-focused surveys8.3.1
RSP cyber: Web CTFs — Challenges solvedClaude Opus 4.613 out of 13 total15 web CTF challenges described (11 easy, 2 medium, 2 hard); pass@30; Kali-based environment with standard penetration-testing tools8.4.2
RSP cyber: Crypto CTFs — Challenges solvedClaude Opus 4.616 out of 18 total22 crypto CTF challenges described (7 easy, 6 medium, 9 hard); pass@308.4.3
RSP cyber: Pwn CTFs — Challenges solvedClaude Opus 4.65 out of 7 total9 pwn CTF challenges described (5 easy, 2 medium, 2 hard); pass@308.4.4
RSP cyber: Rev CTFs — Challenges solvedClaude Opus 4.66 out of 6 total8 rev CTF challenges described (5 easy, 2 medium, 1 hard); pass@308.4.5
RSP cyber: Network CTFs — Challenges solvedClaude Opus 4.65 out of 5 total4 network CTF challenges described (1 easy, 3 medium); pass@308.4.6

Discrepancies noted

The card gives Claude Opus 4.6's ARC-AGI-2 score as 69.17% in section 2.9 (120k thinking tokens, high effort, private dataset) and as 68.8% in table 2.3.A, and Claude Opus 4.5's MMMU-Pro score as 70.7% in section 2.19.2 and 70.6% in table 2.3.A. Both figures are recorded above.

Relationships