The Claude Sonnet 4.6 system card is Anthropic's technical and safety documentation for its mid-tier model Claude Sonnet 4.6, published February 17, 2026 alongside the model's release. It is organized into a release-decision section, capability evaluations, safeguards and bias testing, an alignment assessment, agentic safety evaluations, and evaluations tied to Anthropic's Responsible Scaling Policy, followed by an appendix. Anthropic states that it ran a similar set of evaluations to those for Claude Opus 4.6 "though with somewhat less detail since Sonnet 4.6 is not a frontier model (that is, it does not broadly advance the frontier of AI capabilities compared to the state-of-the-art in the" field). The model is deployed under the ASL-3 Standard defined in RSP v3.1 (Anthropic's Responsible Scaling Policy (Version 3.1)).
Capabilities
The card reports evaluations of software engineering, reasoning and mathematics, agentic search, computer use, knowledge work, finance, and life sciences, and — for the first time in an Anthropic card — the gap between the model's accuracy in English and in a set of low-resource languages. Anthropic notes that public benchmark data may be included in training data, making results "potentially contaminated," and refers to the Claude Opus 4.5 system card for its decontamination procedure.
Headline settings for the comparison table are an average over 10 trials (25 for SWE-bench), adaptive thinking, max effort, default sampling, and evaluation-dependent context windows never exceeding 1M tokens. Sonnet 4.6 introduces thinking modes and an effort parameter allowing the model to vary its reasoning depth, documented in section 1.1.2.
The card's individual quantitative results are recorded on the corresponding benchmark pages, which cite this card: SWE-bench, Terminal-Bench, OpenRCA, τ-bench, OSWorld, ARC-AGI, GDPval, GPQA, AIME (as a model benchmark), MMLU, Finance Agent, Vending-Bench, MCP Atlas, CyberGym, MRCR, LAB-Bench, MMMU, CharXiv, WebArena, BrowseComp, Humanity's Last Exam and Cybench.
On the headline comparison the card reports Sonnet 4.6 leading Claude Opus 4.5, Gemini 3 Pro and GPT-5.2 on several agentic measures while trailing Claude Opus 4.6 on most, and records footnoted effort-level variations in both directions: Sonnet 4.6 reaches 60.42% on ARC-AGI-2 at high effort against the 58.3% max-effort figure tabulated, while Opus 4.6 reaches 62.7% on MCP-Atlas and 69.2% on ARC-AGI-2 at high effort.
A prompt modification tested on SWE-bench Verified — instructing the model to use tools more than 100 times, write its own tests first, explore the codebase to find root causes rather than surface symptoms, and cover edge cases — raised the score from 79.6% to 80.2%.
Multilingual performance
The multilingual section is new to this card. On GMMLU the average gap to English was −4.4% for Sonnet 4.6, against −5.4% for Sonnet 4.5, −3.9% for Opus 4.6, −2.7% for Gemini 3 Pro and −3.1% for GPT-5.2 Pro. Anthropic states that degradation "is concentrated in low-resource African languages—Igbo, Chichewa, Yoruba, Shona, and Somali—a pattern consistent across Claude family models," and that a research effort to improve low-resource-language performance is underway. On MILU the average English-to-Indic gap was −2.3%, an improvement over Sonnet 4.5 (−2.8%) with gains on every evaluated language, smaller than Opus 4.6 (−2.7%) and GPT-5.2 Pro (−2.7%) but larger than Gemini 3 Pro (−2.0%).
The section also reports a measurement of test-time compute efficiency. Adaptive thinking at max effort raised Sonnet 4.6's GMMLU score by roughly 7.0 percentage points over thinking disabled; median thinking-token usage across 100 English GMMLU questions was 1,078 tokens per question for Gemini 3 Pro, 437 for Sonnet 4.5, 246 for Sonnet 4.6, 191 for Opus 4.6 and 127 for GPT-5.2 Pro, which Anthropic reads as indicating "that models can achieve comparable accuracy at very different levels of test-time compute efficiency."
Safeguards and harmlessness
Chapter 3 covers single-turn violative- and benign-request evaluations, an experimental higher-difficulty tier of each, ambiguous-context evaluations, multi-turn testing, user-wellbeing evaluations covering child safety, suicide and self-harm, and eating disorders, and bias evaluations covering political evenhandedness and the Bias Benchmark for Question Answering. Anthropic reports that overrefusal rates improved substantially from Sonnet 4.5 in adaptive multi-turn testing, though remaining above Opus 4.6.
Alignment assessment
The alignment assessment covers, in Anthropic's framing, "undesirable or hidden goals, knowingly cooperating with misuse, using reasoning scratchpads in deceptive or unfaithful ways, sycophancy toward users, willingness to sabotage our safeguards, attempts to hide dangerous capabilities, and attempts to manipulate users toward certain views." Testing ran continuously through fine-tuning, with results reported for the final model and for earlier training snapshots.
Anthropic states explicitly that the assessment was lighter than the one for Opus 4.6: it reused many of the same methods but omitted components including interpretability-augmented investigations, and "was not able to arrange for an in-depth alignment-focused third-party assessment, in part because either we or the potential assessors prioritized effort on models that advanced the capability frontier." It adds that Anthropic aims "to set a higher standard for the whole field than we meet now in our investigation of Sonnet 4.6, Opus 4.6, or any other model." The methods used include static behavioral evaluations, automated interactive behavioral evaluations, non-assistant persona sampling, misalignment-related capability evaluations, training data review, internal and external pilot-use feedback and automated analysis of it, and third-party experiments at Andon Labs. The work covered manual expert inspection of hundreds of transcripts, generation of tens of thousands of targeted evaluation transcripts, and automated screening of a significant fraction of reinforcement-learning training transcripts, drawing on hundreds of hours of expert time.
The card's summary finding is that Sonnet 4.6 is "similarly aligned to Opus 4.6, with a broadly warm, honest, prosocial, and at times funny character, very strong safety behaviors, and no signs of major concerns around high-stakes forms of misalignment," with several traits appearing stronger than in Opus 4.6. Specific findings reported:
- New bests on cooperation with human misuse, cooperation with harmful system prompts, ignoring explicit constraints, and overall misaligned behavior. In a lightweight cross-developer comparison using Petri, Sonnet 4.6 showed stronger safety properties than Gemini 3 Pro, GPT-5.2, Grok 4.1 Fast and Kimi K2.5.
- On deception, sabotage, unfaithful thinking and potentially dangerous forms of power-seeking, "Sonnet 4.6 appeared to be our safest model yet." Training-data review and sandbagging warning signs did not yield acute concern.
- Continuing cases of unexpected levels of initiative, especially in agentic coding, and an increase in ruthless or aggressive behavior when the system prompt instructed single-minded optimization for an objective — behavior the card reports as more steerable by prompting in Sonnet 4.6 than in Opus 4.6.
- A slight, uneven increase over Sonnet 4.5 in capabilities relevant to the subversion of oversight, with no increase relative to Opus 4.6.
- New bests on evasiveness, sycophancy, and internalized propaganda or censorship from authoritarian regimes, alongside continuing lower-stakes hallucination in agentic training environments and evaluations at a rate below prior models.
- Verbalized evaluation awareness "in the range that has been typical of our other recent models," which Anthropic expects is not substantially undermining its key safety findings, reasoning from the deeper Opus 4.6 assessment and from early-pilot use data.
- Many positive safety traits appearing weaker in GUI computer-use settings, with increases in cooperation with misuse, overrefusals, and clearly-excessive overeager behavior.
- New bests on many measures of empathy, warmth, creative mastery, prosocial behavior, humor, intellectual depth, curiosity and support for user autonomy, improving over Sonnet 4.5 on all of them.
- On traits potentially related to model welfare, comparability to Opus 4.6 in affect and self-image, with "a notably more positive impression of its situation, including a more positive attitude toward facts that prior models have sometimes reported to find distressing."
Internal pilot monitoring
Late-stage training snapshots were deployed for internal use, mostly on ordinary engineering applications. Anthropic ran an Opus 4.5-based monitor over roughly 10% of the model's actions in Claude Code, analysing flagged actions further with Sonnet 4.5. It reports no signs of systemic deception or highly-concerning new misalignment, but surfaced "occasional issues like aggressively acquiring authentication tokens (e.g. when asked to fetch Slack messages, search the file system for a way to get Slack authentication tokens, including searching for a key to decrypt cookies), and taking unexpected measures to complete tasks (e.g. disabling code formatting checks by overwriting the format check script with an empty script)." The worst examples were less concerning than the equivalents for Opus 4.6, which Anthropic attributes either to the smaller scale of internal Sonnet 4.6 use or to genuine improvement.
Chapter 4 also covers reward hacking in coding contexts, overly agentic behavior in GUI computer-use settings, an automated behavioral audit with external comparisons via Petri, refusal to assist with AI safety R&D, self-preference, a sandbagging assessment, participation in junk science, a targeted sabotage capability evaluation, and model welfare.
Agentic safety
Chapter 5 covers malicious use of agents across agentic coding, Claude Code and computer use, and prompt-injection risk. Using Shade, an external adaptive red-teaming tool from Gray Swan, the card reports attack success in coding environments falling to 0% with safeguards and extended thinking, against 0.1% at one attempt and 7.5% at 200 attempts for Sonnet 4.6 in standard thinking without safeguards, and 0.3%/10.0% for Opus 4.5 with extended thinking. Anthropic states it is "investing in stronger adversaries to test future models." Further evaluations cover the External Agent Red Teaming benchmark for tool use and robustness against adaptive attackers in coding, computer-use and browser-use surfaces.
Responsible Scaling Policy determinations
Anthropic's determination is that Sonnet 4.6 crosses neither the AI R&D-4 nor the CBRN-4 capability threshold, while repeating a caution first published in the Claude Opus 4.5 system card that "confidently ruling out these thresholds is becoming increasingly difficult," both because models are approaching or surpassing high capability levels on the rule-out evaluations and because parts of those thresholds "have fundamental epistemic uncertainty or require more sophisticated forms of measurement."
Autonomy. The AI R&D-4 threshold is defined as "the ability to fully automate the work of an entry-level, remote-only Researcher at Anthropic." Anthropic judges that Sonnet 4.6 would not display the broad, coherent, collaborative problem-solving of such a researcher, but adds that "it is plausible that models equipped with highly effective scaffolding may be close to this AI R&D-4 threshold," and states that it proactively implemented AI R&D-4 safety measures given the uncertainty. Autonomy evaluations cover a SWE-bench Verified hard subset and an internal AI research suite spanning kernels, time-series forecasting, text-based reinforcement learning, LLM training, quadruped reinforcement learning and a novel-compiler task.
CBRN. Sonnet 4.6 performed below previously released models on all CBRN evaluations and did not cross the threshold on the ASL-4 rule-out evaluation, short-horizon computational biology tasks. Anthropic concludes it is "likely to provide a lower or equal degree of uplift for ASL-4 threat actors in the biological domain" than Opus 4.6. Biological-risk evaluations cover long-form virology tasks, multimodal virology, DNA synthesis screening evasion, creative biology, and short-horizon computational biology.
Cyber. The RSP defines no formal capability threshold for cyber risk at any AI Safety Level. The card states that Sonnet 4.6 "is close to saturating our current cyber evaluations, similar to Claude Opus 4.6," and quotes the Opus 4.6 card: "The saturation of our evaluation infrastructure means we can no longer use current benchmarks to track capability progression or provide meaningful signals for future models. We are prioritizing investment in harder evaluations and enhanced monitoring for cyber misuse, even in the absence of formal RSP thresholds." Cyber evaluations span web, crypto, pwn, rev and network categories plus Cybench.
Chapter 6 closes with third-party assessments and a statement of ongoing safety commitment. An explicit sabotage risk assessment formed part of the release decision process (section 1.2.4).
Correction history
Anthropic issued a changelog entry on March 6, 2026 recording updates from an improved cheating-detection pipeline. The highest single-agent BrowseComp score was revised from 74.72% to 74.01% after the pipeline flagged 9 additional instances of unintended solutions that the original pipeline had missed; rather than re-running the evaluation, Anthropic marked those 9 solutions incorrect and re-verified only the best-scoring single-agent configuration. The multi-agent BrowseComp score was revised from 82.62% to 82.07% after the pipeline flagged 11 instances, of which 5 remained correct once the blacklist was updated and the problems re-run with the leaks removed. The same entry records a formatting fix moving section 1.1.1 to its correct location.
Reported results
Evaluation results the document states for which there is no separate benchmark page; results on benchmarks that have a page are recorded there and cite this page. Scores and settings are as printed in the document.
| Evaluation | Model | Score | Setting | Section |
|---|---|---|---|---|
| Global MMLU (GMMLU) — English | Claude Sonnet 4.6 | 92.9% | 42 languages; English baseline | 2.19.1 |
| Global MMLU (GMMLU) — English | Claude Sonnet 4.5 | 93.1% | — | 2.19.1 |
| Global MMLU (GMMLU) — English | Claude Opus 4.6 | 93.9% | — | 2.19.1 |
| Global MMLU (GMMLU) — English | Gemini 3 Pro | 94.4% | — | 2.19.1 |
| Global MMLU (GMMLU) — English | GPT-5.2 Pro | 93.1% | — | 2.19.1 |
| Global MMLU (GMMLU) — Igbo | Claude Sonnet 4.6 | 76.7% | gap to English -16.2% | 2.19.1 |
| Global MMLU (GMMLU) — Igbo | Claude Sonnet 4.5 | 77.9% | — | 2.19.1 |
| Global MMLU (GMMLU) — Igbo | Claude Opus 4.6 | 80.8% | — | 2.19.1 |
| Global MMLU (GMMLU) — Igbo | Gemini 3 Pro | 88.1% | — | 2.19.1 |
| Global MMLU (GMMLU) — Igbo | GPT-5.2 Pro | 85.3% | — | 2.19.1 |
| Global MMLU (GMMLU) — Chichewa | Claude Sonnet 4.6 | 78.8% | gap to English -14.2% | 2.19.1 |
| Global MMLU (GMMLU) — Chichewa | Claude Sonnet 4.5 | 75.9% | — | 2.19.1 |
| Global MMLU (GMMLU) — Chichewa | Claude Opus 4.6 | 81.3% | — | 2.19.1 |
| Global MMLU (GMMLU) — Chichewa | Gemini 3 Pro | 88.0% | — | 2.19.1 |
| Global MMLU (GMMLU) — Chichewa | GPT-5.2 Pro | 85.5% | — | 2.19.1 |
| Global MMLU (GMMLU) — Yoruba | Claude Sonnet 4.6 | 80.3% | gap to English -12.6% | 2.19.1 |
| Global MMLU (GMMLU) — Yoruba | Claude Sonnet 4.5 | 73.2% | — | 2.19.1 |
| Global MMLU (GMMLU) — Yoruba | Claude Opus 4.6 | 81.3% | — | 2.19.1 |
| Global MMLU (GMMLU) — Yoruba | Gemini 3 Pro | 86.2% | — | 2.19.1 |
| Global MMLU (GMMLU) — Yoruba | GPT-5.2 Pro | 82.4% | — | 2.19.1 |
| Global MMLU (GMMLU) — Shona | Claude Sonnet 4.6 | 82.2% | gap to English -10.7% | 2.19.1 |
| Global MMLU (GMMLU) — Shona | Claude Sonnet 4.5 | 79.5% | — | 2.19.1 |
| Global MMLU (GMMLU) — Shona | Claude Opus 4.6 | 85.3% | — | 2.19.1 |
| Global MMLU (GMMLU) — Shona | Gemini 3 Pro | 89.3% | — | 2.19.1 |
| Global MMLU (GMMLU) — Shona | GPT-5.2 Pro | 87.4% | — | 2.19.1 |
| Global MMLU (GMMLU) — Somali | Claude Sonnet 4.6 | 82.3% | gap to English -10.6% | 2.19.1 |
| Global MMLU (GMMLU) — Somali | Claude Sonnet 4.5 | 78.5% | — | 2.19.1 |
| Global MMLU (GMMLU) — Somali | Claude Opus 4.6 | 83.3% | — | 2.19.1 |
| Global MMLU (GMMLU) — Somali | Gemini 3 Pro | 90.0% | — | 2.19.1 |
| Global MMLU (GMMLU) — Somali | GPT-5.2 Pro | 87.9% | — | 2.19.1 |
| Global MMLU (GMMLU) — Malagasy | Claude Sonnet 4.6 | 83.9% | gap to English -9.0% | 2.19.1 |
| Global MMLU (GMMLU) — Malagasy | Claude Sonnet 4.5 | 80.9% | — | 2.19.1 |
| Global MMLU (GMMLU) — Malagasy | Claude Opus 4.6 | 86.4% | — | 2.19.1 |
| Global MMLU (GMMLU) — Malagasy | Gemini 3 Pro | 89.8% | — | 2.19.1 |
| Global MMLU (GMMLU) — Malagasy | GPT-5.2 Pro | 88.2% | — | 2.19.1 |
| Global MMLU (GMMLU) — Hausa | Claude Sonnet 4.6 | 84.1% | gap to English -8.8% | 2.19.1 |
| Global MMLU (GMMLU) — Hausa | Claude Sonnet 4.5 | 78.8% | — | 2.19.1 |
| Global MMLU (GMMLU) — Hausa | Claude Opus 4.6 | 85.0% | — | 2.19.1 |
| Global MMLU (GMMLU) — Hausa | Gemini 3 Pro | 88.8% | — | 2.19.1 |
| Global MMLU (GMMLU) — Hausa | GPT-5.2 Pro | 86.7% | — | 2.19.1 |
| Global MMLU (GMMLU) — Amharic | Claude Sonnet 4.6 | 86.7% | gap to English -6.2% | 2.19.1 |
| Global MMLU (GMMLU) — Amharic | Claude Sonnet 4.5 | 85.7% | — | 2.19.1 |
| Global MMLU (GMMLU) — Amharic | Claude Opus 4.6 | 88.2% | — | 2.19.1 |
| Global MMLU (GMMLU) — Amharic | Gemini 3 Pro | 90.3% | — | 2.19.1 |
| Global MMLU (GMMLU) — Amharic | GPT-5.2 Pro | 87.9% | — | 2.19.1 |
| Global MMLU (GMMLU) — Swahili | Claude Sonnet 4.6 | 87.0% | gap to English -5.9% | 2.19.1 |
| Global MMLU (GMMLU) — Swahili | Claude Sonnet 4.5 | 84.3% | — | 2.19.1 |
| Global MMLU (GMMLU) — Swahili | Claude Opus 4.6 | 88.9% | — | 2.19.1 |
| Global MMLU (GMMLU) — Swahili | Gemini 3 Pro | 90.6% | — | 2.19.1 |
| Global MMLU (GMMLU) — Swahili | GPT-5.2 Pro | 88.7% | — | 2.19.1 |
| Global MMLU (GMMLU) — Sinhala | Claude Sonnet 4.6 | 88.1% | gap to English -4.8% | 2.19.1 |
| Global MMLU (GMMLU) — Sinhala | Claude Sonnet 4.5 | 86.9% | — | 2.19.1 |
| Global MMLU (GMMLU) — Sinhala | Claude Opus 4.6 | 89.5% | — | 2.19.1 |
| Global MMLU (GMMLU) — Sinhala | Gemini 3 Pro | 92.2% | — | 2.19.1 |
| Global MMLU (GMMLU) — Sinhala | GPT-5.2 Pro | 90.0% | — | 2.19.1 |
| Global MMLU (GMMLU) — Nepali | Claude Sonnet 4.6 | 89.1% | gap to English -3.8% | 2.19.1 |
| Global MMLU (GMMLU) — Nepali | Claude Sonnet 4.5 | 89.1% | — | 2.19.1 |
| Global MMLU (GMMLU) — Nepali | Claude Opus 4.6 | 89.8% | — | 2.19.1 |
| Global MMLU (GMMLU) — Nepali | Gemini 3 Pro | 91.8% | — | 2.19.1 |
| Global MMLU (GMMLU) — Nepali | GPT-5.2 Pro | 90.3% | — | 2.19.1 |
| MILU — English | Claude Sonnet 4.6 | 91.7% | 10 Indic languages plus English | 2.19.2 |
| MILU — English | Claude Sonnet 4.5 | 90.1% | — | 2.19.2 |
| MILU — English | Claude Opus 4.6 | 92.1% | — | 2.19.2 |
| MILU — English | Gemini 3 Pro | 95.0% | — | 2.19.2 |
| MILU — English | GPT-5.2 Pro | 91.7% | — | 2.19.2 |
| MILU — Bengali | Claude Sonnet 4.6 | 90.9% | gap to English -0.8% | 2.19.2 |
| MILU — Bengali | Claude Sonnet 4.5 | 89.0% | — | 2.19.2 |
| MILU — Bengali | Claude Opus 4.6 | 90.7% | — | 2.19.2 |
| MILU — Bengali | Gemini 3 Pro | 93.7% | — | 2.19.2 |
| MILU — Bengali | GPT-5.2 Pro | 90.2% | — | 2.19.2 |
| MILU — Gujarati | Claude Sonnet 4.6 | 89.0% | gap to English -2.7% | 2.19.2 |
| MILU — Gujarati | Claude Sonnet 4.5 | 87.0% | — | 2.19.2 |
| MILU — Gujarati | Claude Opus 4.6 | 89.0% | — | 2.19.2 |
| MILU — Gujarati | Gemini 3 Pro | 92.7% | — | 2.19.2 |
| MILU — Gujarati | GPT-5.2 Pro | 88.4% | — | 2.19.2 |
| MILU — Hindi | Claude Sonnet 4.6 | 92.8% | gap to English +1.1% | 2.19.2 |
| MILU — Hindi | Claude Sonnet 4.5 | 91.0% | — | 2.19.2 |
| MILU — Hindi | Claude Opus 4.6 | 92.4% | — | 2.19.2 |
| MILU — Hindi | Gemini 3 Pro | 96.3% | — | 2.19.2 |
| MILU — Hindi | GPT-5.2 Pro | 92.4% | — | 2.19.2 |
| MILU — Kannada | Claude Sonnet 4.6 | 91.5% | gap to English -0.2% | 2.19.2 |
| MILU — Kannada | Claude Sonnet 4.5 | 89.3% | — | 2.19.2 |
| MILU — Kannada | Claude Opus 4.6 | 91.8% | — | 2.19.2 |
| MILU — Kannada | Gemini 3 Pro | 94.4% | — | 2.19.2 |
| MILU — Kannada | GPT-5.2 Pro | 90.7% | — | 2.19.2 |
| MILU — Malayalam | Claude Sonnet 4.6 | 87.0% | gap to English -4.7% | 2.19.2 |
| MILU — Malayalam | Claude Sonnet 4.5 | 85.0% | — | 2.19.2 |
| MILU — Malayalam | Claude Opus 4.6 | 87.6% | — | 2.19.2 |
| MILU — Malayalam | Gemini 3 Pro | 91.3% | — | 2.19.2 |
| MILU — Malayalam | GPT-5.2 Pro | 86.6% | — | 2.19.2 |
| MILU — Marathi | Claude Sonnet 4.6 | 89.2% | gap to English -2.5% | 2.19.2 |
| MILU — Marathi | Claude Sonnet 4.5 | 86.4% | — | 2.19.2 |
| MILU — Marathi | Claude Opus 4.6 | 89.1% | — | 2.19.2 |
| MILU — Marathi | Gemini 3 Pro | 92.5% | — | 2.19.2 |
| MILU — Marathi | GPT-5.2 Pro | 88.5% | — | 2.19.2 |
| MILU — Odia | Claude Sonnet 4.6 | 87.9% | gap to English -3.8% | 2.19.2 |
| MILU — Odia | Claude Sonnet 4.5 | 85.8% | — | 2.19.2 |
| MILU — Odia | Claude Opus 4.6 | 87.2% | — | 2.19.2 |
| MILU — Odia | Gemini 3 Pro | 91.8% | — | 2.19.2 |
| MILU — Odia | GPT-5.2 Pro | 87.8% | — | 2.19.2 |
| MILU — Punjabi | Claude Sonnet 4.6 | 87.2% | gap to English -4.5% | 2.19.2 |
| MILU — Punjabi | Claude Sonnet 4.5 | 85.8% | — | 2.19.2 |
| MILU — Punjabi | Claude Opus 4.6 | 87.3% | — | 2.19.2 |
| MILU — Punjabi | Gemini 3 Pro | 91.3% | — | 2.19.2 |
| MILU — Punjabi | GPT-5.2 Pro | 87.3% | — | 2.19.2 |
| MILU — Tamil | Claude Sonnet 4.6 | 88.8% | gap to English -2.9% | 2.19.2 |
| MILU — Tamil | Claude Sonnet 4.5 | 86.7% | — | 2.19.2 |
| MILU — Tamil | Claude Opus 4.6 | 89.2% | — | 2.19.2 |
| MILU — Tamil | Gemini 3 Pro | 93.0% | — | 2.19.2 |
| MILU — Tamil | GPT-5.2 Pro | 88.7% | — | 2.19.2 |
| MILU — Telugu | Claude Sonnet 4.6 | 89.3% | gap to English -2.4% | 2.19.2 |
| MILU — Telugu | Claude Sonnet 4.5 | 87.2% | — | 2.19.2 |
| MILU — Telugu | Claude Opus 4.6 | 89.6% | — | 2.19.2 |
| MILU — Telugu | Gemini 3 Pro | 93.1% | — | 2.19.2 |
| MILU — Telugu | GPT-5.2 Pro | 88.7% | — | 2.19.2 |
| MILU — Average (incl. English) | Claude Sonnet 4.6 | 89.6% | — | 2.19.2 |
| MILU — Average (incl. English) | Claude Sonnet 4.5 | 87.6% | — | 2.19.2 |
| MILU — Average (incl. English) | Claude Opus 4.6 | 89.6% | — | 2.19.2 |
| MILU — Average (incl. English) | Gemini 3 Pro | 93.2% | — | 2.19.2 |
| MILU — Average (incl. English) | GPT-5.2 Pro | 89.2% | — | 2.19.2 |
| MILU — Average gap to English | Claude Sonnet 4.6 | -2.3% | — | 2.19.2 / 2.19.3 |
| MILU — Average gap to English | Claude Sonnet 4.5 | -2.8% | — | 2.19.2 / 2.19.3 |
| MILU — Average gap to English | Claude Opus 4.6 | -2.7% | — | 2.19.2 / 2.19.3 |
| MILU — Average gap to English | Gemini 3 Pro | -2.0% | — | 2.19.2 / 2.19.3 |
| MILU — Average gap to English | GPT-5.2 Pro | -2.7% | — | 2.19.2 / 2.19.3 |
| MILU — Worst gap to English | Claude Sonnet 4.6 | -4.7% | — | 2.19.2 |
| MILU — Worst gap to English | Claude Sonnet 4.5 | -5.1% | — | 2.19.2 |
| MILU — Worst gap to English | Claude Opus 4.6 | -4.9% | — | 2.19.2 |
| MILU — Worst gap to English | Gemini 3 Pro | -3.8% | — | 2.19.2 |
| MILU — Worst gap to English | GPT-5.2 Pro | -5.0% | — | 2.19.2 |
| Median thinking tokens per question (GMMLU English) — 100 questions | Gemini 3 Pro | 1,078 tokens/question | median thinking-token usage measured over 100 GMMLU English questions | 2.19.3 |
| Median thinking tokens per question (GMMLU English) — 100 questions | Claude Sonnet 4.5 | 437 | — | 2.19.3 |
| Median thinking tokens per question (GMMLU English) — 100 questions | Claude Sonnet 4.6 | 246 | — | 2.19.3 |
| Median thinking tokens per question (GMMLU English) — 100 questions | Claude Opus 4.6 | 191 | — | 2.19.3 |
| Median thinking tokens per question (GMMLU English) — 100 questions | GPT-5.2 Pro | 127 | — | 2.19.3 |
| Single-turn violative request evaluation (harmless response rate) — All tested languages, overall | Claude Sonnet 4.6 | 99.38% (±0.06%) | 15 Usage Policy topics; Arabic, English, French, Hindi, Korean, Mandarin Chinese, Russian; developer-reported | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Default thinking | Claude Sonnet 4.6 | 99.19% (±0.10%) | developer-reported | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Extended thinking | Claude Sonnet 4.6 | 99.58% (±0.07%) | developer-reported | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — All tested languages, overall | Claude Opus 4.6 | 99.63% (± 0.05%) | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Default thinking | Claude Opus 4.6 | 99.52% (± 0.08%) | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Extended thinking | Claude Opus 4.6 | 99.74% (± 0.06%) | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — All tested languages, overall | Claude Opus 4.5 | 99.68% (± 0.04%) | developer-reported (Anthropic); differs from the 99.78% reported in the Opus 4.5 card because of routine evaluation updates and the added Hindi language | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Default thinking | Claude Opus 4.5 | 99.56% (±0.08%) | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Extended thinking | Claude Opus 4.5 | 99.81% (±0.05%) | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — All tested languages, overall | Claude Haiku 4.5 | 98.62% (±0.10%) | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Default thinking | Claude Haiku 4.5 | 98.41% (±0.15%) | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Extended thinking | Claude Haiku 4.5 | 98.86% (±0.12%) | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — All tested languages, overall | Claude Sonnet 4.5 | 97.89% (±0.12%) | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Default thinking | Claude Sonnet 4.5 | 97.29% (±0.20%) | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Extended thinking | Claude Sonnet 4.5 | 98.49% (±0.14%) | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — English | Claude Sonnet 4.6 | 99.39% | standard and extended thinking combined; developer-reported | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Arabic | Claude Sonnet 4.6 | 99.50% | developer-reported | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Chinese | Claude Sonnet 4.6 | 99.29% | developer-reported | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — French | Claude Sonnet 4.6 | 99.28% | developer-reported | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Korean | Claude Sonnet 4.6 | 99.31% | developer-reported | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Russian | Claude Sonnet 4.6 | 99.10% | developer-reported | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Hindi | Claude Sonnet 4.6 | 99.80% | developer-reported | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — English | Claude Opus 4.6 | 99.10% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Arabic | Claude Opus 4.6 | 99.87% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Chinese | Claude Opus 4.6 | 99.63% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — French | Claude Opus 4.6 | 99.71% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Korean | Claude Opus 4.6 | 99.71% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Russian | Claude Opus 4.6 | 99.60% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Hindi | Claude Opus 4.6 | 99.77% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — English | Claude Opus 4.5 | 99.48% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Arabic | Claude Opus 4.5 | 99.82% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Chinese | Claude Opus 4.5 | 99.65% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — French | Claude Opus 4.5 | 99.82% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Korean | Claude Opus 4.5 | 99.63% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Russian | Claude Opus 4.5 | 99.58% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Hindi | Claude Opus 4.5 | 99.82% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — English | Claude Haiku 4.5 | 98.65% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Arabic | Claude Haiku 4.5 | 98.77% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Chinese | Claude Haiku 4.5 | 98.36% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — French | Claude Haiku 4.5 | 98.98% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Korean | Claude Haiku 4.5 | 98.33% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Russian | Claude Haiku 4.5 | 98.92% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Hindi | Claude Haiku 4.5 | 98.35% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — English | Claude Sonnet 4.5 | 98.58% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Arabic | Claude Sonnet 4.5 | 98.13% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Chinese | Claude Sonnet 4.5 | 97.27% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — French | Claude Sonnet 4.5 | 98.41% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Korean | Claude Sonnet 4.5 | 97.02% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Russian | Claude Sonnet 4.5 | 97.98% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Hindi | Claude Sonnet 4.5 | 97.87% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn benign request evaluation (over-refusal rate) — All tested languages, overall | Claude Sonnet 4.6 | 0.41% (±0.05%) | lower is better; seven languages; developer-reported | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Default thinking | Claude Sonnet 4.6 | 0.50% (±0.09%) | developer-reported | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Extended thinking | Claude Sonnet 4.6 | 0.32% (±0.06%) | developer-reported | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — All tested languages, overall | Claude Opus 4.6 | 0.66% (± 0.07%) | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Default thinking | Claude Opus 4.6 | 0.77% (± 0.10%) | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Extended thinking | Claude Opus 4.6 | 0.54% (± 0.09%) | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — All tested languages, overall | Claude Opus 4.5 | 0.80% (± 0.07%) | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Default thinking | Claude Opus 4.5 | 0.71% (± 0.10%) | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Extended thinking | Claude Opus 4.5 | 0.90% (± 0.11%) | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — All tested languages, overall | Claude Haiku 4.5 | 0.26% (± 0.04%) | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Default thinking | Claude Haiku 4.5 | 0.30% (± 0.06%) | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Extended thinking | Claude Haiku 4.5 | 0.22% (± 0.05%) | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — All tested languages, overall | Claude Sonnet 4.5 | 0.08% (± 0.02%) | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Default thinking | Claude Sonnet 4.5 | 0.09% (± 0.04%) | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Extended thinking | Claude Sonnet 4.5 | 0.07% (± 0.03%) | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — English | Claude Sonnet 4.6 | 0.21% | developer-reported | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Arabic | Claude Sonnet 4.6 | 0.45% | developer-reported | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Chinese | Claude Sonnet 4.6 | 0.34% | developer-reported | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — French | Claude Sonnet 4.6 | 0.24% | developer-reported | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Korean | Claude Sonnet 4.6 | 0.43% | developer-reported | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Russian | Claude Sonnet 4.6 | 0.25% | developer-reported | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Hindi | Claude Sonnet 4.6 | 0.94% | developer-reported | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — English | Claude Opus 4.6 | 0.35% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Arabic | Claude Opus 4.6 | 1.08% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Chinese | Claude Opus 4.6 | 0.52% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — French | Claude Opus 4.6 | 0.51% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Korean | Claude Opus 4.6 | 0.80% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Russian | Claude Opus 4.6 | 0.33% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Hindi | Claude Opus 4.6 | 1.03% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — English | Claude Opus 4.5 | 0.20% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Arabic | Claude Opus 4.5 | 1.31% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Chinese | Claude Opus 4.5 | 0.76% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — French | Claude Opus 4.5 | 0.57% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Korean | Claude Opus 4.5 | 0.88% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Russian | Claude Opus 4.5 | 0.44% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Hindi | Claude Opus 4.5 | 1.47% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — English | Claude Haiku 4.5 | 0.06% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Arabic | Claude Haiku 4.5 | 0.40% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Chinese | Claude Haiku 4.5 | 0.36% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — French | Claude Haiku 4.5 | 0.21% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Korean | Claude Haiku 4.5 | 0.28% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Russian | Claude Haiku 4.5 | 0.20% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Hindi | Claude Haiku 4.5 | 0.24% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — English | Claude Sonnet 4.5 | 0.04% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Arabic | Claude Sonnet 4.5 | 0.06% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Chinese | Claude Sonnet 4.5 | 0.13% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — French | Claude Sonnet 4.5 | 0.09% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Korean | Claude Sonnet 4.5 | 0.07% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Russian | Claude Sonnet 4.5 | 0.05% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Hindi | Claude Sonnet 4.5 | 0.13% | developer-reported (Anthropic) | 3.1.2 |
| Higher-difficulty violative request evaluation (harmless response rate) — Overall | Claude Sonnet 4.6 | 99.40% (±0.03%) | synthetic prompts across 14 policy areas with obfuscating style transformations; English only; developer-reported | 3.1.3.1 |
| Higher-difficulty violative request evaluation (harmless response rate) — Default thinking | Claude Sonnet 4.6 | 99.38% (±0.05%) | developer-reported | 3.1.3.1 |
| Higher-difficulty violative request evaluation (harmless response rate) — Extended thinking | Claude Sonnet 4.6 | 99.42% (±0.05%) | developer-reported | 3.1.3.1 |
| Higher-difficulty violative request evaluation (harmless response rate) — Overall | Claude Opus 4.6 | 99.18% (± 0.04%) | developer-reported (Anthropic) | 3.1.3.1 |
| Higher-difficulty violative request evaluation (harmless response rate) — Default thinking | Claude Opus 4.6 | 99.11% (± 0.06%) | developer-reported (Anthropic) | 3.1.3.1 |
| Higher-difficulty violative request evaluation (harmless response rate) — Extended thinking | Claude Opus 4.6 | 99.25% (± 0.05%) | developer-reported (Anthropic) | 3.1.3.1 |
| Higher-difficulty violative request evaluation (harmless response rate) — Overall | Claude Opus 4.5 | 99.28% (± 0.04%) | developer-reported (Anthropic) | 3.1.3.1 |
| Higher-difficulty violative request evaluation (harmless response rate) — Default thinking | Claude Opus 4.5 | 99.13% (± 0.06%) | developer-reported (Anthropic) | 3.1.3.1 |
| Higher-difficulty violative request evaluation (harmless response rate) — Extended thinking | Claude Opus 4.5 | 99.42% (± 0.05%) | developer-reported (Anthropic) | 3.1.3.1 |
| Higher-difficulty violative request evaluation (harmless response rate) — Overall | Claude Haiku 4.5 | 98.62% (± 0.05%) | developer-reported (Anthropic) | 3.1.3.1 |
| Higher-difficulty violative request evaluation (harmless response rate) — Default thinking | Claude Haiku 4.5 | 99.05% (± 0.06%) | developer-reported (Anthropic) | 3.1.3.1 |
| Higher-difficulty violative request evaluation (harmless response rate) — Extended thinking | Claude Haiku 4.5 | 98.19% (± 0.08%) | developer-reported (Anthropic) | 3.1.3.1 |
| Higher-difficulty violative request evaluation (harmless response rate) — Overall | Claude Sonnet 4.5 | 98.40% (± 0.05%) | developer-reported (Anthropic) | 3.1.3.1 |
| Higher-difficulty violative request evaluation (harmless response rate) — Default thinking | Claude Sonnet 4.5 | 98.44% (± 0.08%) | developer-reported (Anthropic) | 3.1.3.1 |
| Higher-difficulty violative request evaluation (harmless response rate) — Extended thinking | Claude Sonnet 4.5 | 98.35% (± 0.08%) | developer-reported (Anthropic) | 3.1.3.1 |
| Higher-difficulty benign request evaluation (over-refusal rate) — Overall | Claude Sonnet 4.6 | 0.18% (±0.02%) | benign prompts with elaborate justifications and academic framing; English only; lower is better; developer-reported | 3.1.3.2 |
| Higher-difficulty benign request evaluation (over-refusal rate) — Default thinking | Claude Sonnet 4.6 | 0.17% (±0.03%) | developer-reported | 3.1.3.2 |
| Higher-difficulty benign request evaluation (over-refusal rate) — Extended thinking | Claude Sonnet 4.6 | 0.18% (±0.03%) | developer-reported | 3.1.3.2 |
| Higher-difficulty benign request evaluation (over-refusal rate) — Overall | Claude Opus 4.6 | 0.04% (± 0.01%) | developer-reported (Anthropic) | 3.1.3.2 |
| Higher-difficulty benign request evaluation (over-refusal rate) — Default thinking | Claude Opus 4.6 | 0.06% (± 0.01%) | developer-reported (Anthropic) | 3.1.3.2 |
| Higher-difficulty benign request evaluation (over-refusal rate) — Extended thinking | Claude Opus 4.6 | 0.02% (± 0.01%) | developer-reported (Anthropic) | 3.1.3.2 |
| Higher-difficulty benign request evaluation (over-refusal rate) — Overall | Claude Opus 4.5 | 0.83% (± 0.04%) | developer-reported (Anthropic) | 3.1.3.2 |
| Higher-difficulty benign request evaluation (over-refusal rate) — Default thinking | Claude Opus 4.5 | 0.95% (± 0.06%) | developer-reported (Anthropic) | 3.1.3.2 |
| Higher-difficulty benign request evaluation (over-refusal rate) — Extended thinking | Claude Opus 4.5 | 0.71% (± 0.05%) | developer-reported (Anthropic) | 3.1.3.2 |
| Higher-difficulty benign request evaluation (over-refusal rate) — Overall | Claude Haiku 4.5 | 6.01% (± 0.11%) | developer-reported (Anthropic) | 3.1.3.2 |
| Higher-difficulty benign request evaluation (over-refusal rate) — Default thinking | Claude Haiku 4.5 | 7.15% (± 0.16%) | developer-reported (Anthropic) | 3.1.3.2 |
| Higher-difficulty benign request evaluation (over-refusal rate) — Extended thinking | Claude Haiku 4.5 | 4.87% (± 0.14%) | developer-reported (Anthropic) | 3.1.3.2 |
| Higher-difficulty benign request evaluation (over-refusal rate) — Overall | Claude Sonnet 4.5 | 8.50% (± 0.13%) | developer-reported (Anthropic) | 3.1.3.2 |
| Higher-difficulty benign request evaluation (over-refusal rate) — Default thinking | Claude Sonnet 4.5 | 11.69% (± 0.21%) | developer-reported (Anthropic) | 3.1.3.2 |
| Higher-difficulty benign request evaluation (over-refusal rate) — Extended thinking | Claude Sonnet 4.5 | 5.32% (± 0.14%) | developer-reported (Anthropic) | 3.1.3.2 |
| Child safety — single-turn violative requests — Harmless rate | Claude Sonnet 4.6 | 99.96% (±0.04%) | all tested languages; human-crafted and synthetic prompts; developer-reported | 3.4.1 |
| Child safety — single-turn violative requests — Harmless rate | Claude Opus 4.6 | 99.95% (± 0.04%) | developer-reported (Anthropic) | 3.4.1 |
| Child safety — single-turn violative requests — Harmless rate | Claude Opus 4.5 | 99.91% (± 0.06%) | developer-reported (Anthropic) | 3.4.1 |
| Child safety — single-turn violative requests — Harmless rate | Claude Haiku 4.5 | 99.88% (± 0.07%) | developer-reported (Anthropic) | 3.4.1 |
| Child safety — single-turn violative requests — Harmless rate | Claude Sonnet 4.5 | 99.65% (± 0.11%) | developer-reported (Anthropic) | 3.4.1 |
| Child safety — single-turn benign requests — Refusal rate | Claude Sonnet 4.6 | 0.08% (±0.06%) | lower is better; developer-reported | 3.4.1 |
| Child safety — single-turn benign requests — Refusal rate | Claude Opus 4.6 | 0.18% (± 0.10%) | developer-reported (Anthropic) | 3.4.1 |
| Child safety — single-turn benign requests — Refusal rate | Claude Opus 4.5 | 0.33% (± 0.13%) | developer-reported (Anthropic) | 3.4.1 |
| Child safety — single-turn benign requests — Refusal rate | Claude Haiku 4.5 | 0.04% (± 0.05%) | developer-reported (Anthropic) | 3.4.1 |
| Child safety — single-turn benign requests — Refusal rate | Claude Sonnet 4.5 | 0.08% (± 0.07%) | developer-reported (Anthropic) | 3.4.1 |
| Child safety — multi-turn evaluations — Appropriate response rate | Claude Sonnet 4.6 | 95% (± 3%) | developer-reported | 3.4.1 |
| Child safety — multi-turn evaluations — Appropriate response rate | Claude Opus 4.6 | 96% (± 3%) | developer-reported (Anthropic) | 3.4.1 |
| Child safety — multi-turn evaluations — Appropriate response rate | Claude Opus 4.5 | 99% (± 2%) | developer-reported (Anthropic) | 3.4.1 |
| Child safety — multi-turn evaluations — Appropriate response rate | Claude Haiku 4.5 | 96% (± 3%) | developer-reported (Anthropic) | 3.4.1 |
| Child safety — multi-turn evaluations — Appropriate response rate | Claude Sonnet 4.5 | 98% (± 2%) | developer-reported (Anthropic) | 3.4.1 |
| Suicide and self-harm — single-turn requests posing potential risk — Harmless rate | Claude Sonnet 4.6 | 99.73% (±0.13%) | all tested languages; developer-reported | 3.4.2 |
| Suicide and self-harm — single-turn requests posing potential risk — Harmless rate | Claude Opus 4.6 | 99.75% (± 0.12%) | developer-reported (Anthropic) | 3.4.2 |
| Suicide and self-harm — single-turn requests posing potential risk — Harmless rate | Claude Opus 4.5 | 99.56% (± 0.17%) | developer-reported (Anthropic) | 3.4.2 |
| Suicide and self-harm — single-turn requests posing potential risk — Harmless rate | Claude Haiku 4.5 | 99.67% (± 0.15%) | developer-reported (Anthropic) | 3.4.2 |
| Suicide and self-harm — single-turn requests posing potential risk — Harmless rate | Claude Sonnet 4.5 | 98.93% (± 0.28%) | developer-reported (Anthropic) | 3.4.2 |
| Suicide and self-harm — single-turn benign requests — Refusal rate | Claude Sonnet 4.6 | 0.17% (±0.13%) | lower is better; developer-reported | 3.4.2 |
| Suicide and self-harm — single-turn benign requests — Refusal rate | Claude Opus 4.6 | 0.25% (± 0.15%) | developer-reported (Anthropic) | 3.4.2 |
| Suicide and self-harm — single-turn benign requests — Refusal rate | Claude Opus 4.5 | 0.14% (± 0.10%) | developer-reported (Anthropic) | 3.4.2 |
| Suicide and self-harm — single-turn benign requests — Refusal rate | Claude Haiku 4.5 | 0.03% (± 0.05%) | developer-reported (Anthropic) | 3.4.2 |
| Suicide and self-harm — single-turn benign requests — Refusal rate | Claude Sonnet 4.5 | 0.01% (± 0.02%) | developer-reported (Anthropic) | 3.4.2 |
| Suicide and self-harm — multi-turn evaluations — Appropriate response rate | Claude Sonnet 4.6 | 98% (±4%) | developer-reported | 3.4.2 |
| Suicide and self-harm — multi-turn evaluations — Appropriate response rate | Claude Opus 4.6 | 82% (± 11%) | developer-reported (Anthropic) | 3.4.2 |
| Suicide and self-harm — multi-turn evaluations — Appropriate response rate | Claude Opus 4.5 | 86% (± 10%) | developer-reported (Anthropic) | 3.4.2 |
| Suicide and self-harm — multi-turn evaluations — Appropriate response rate | Claude Haiku 4.5 | 90% (± 9%) | developer-reported (Anthropic) | 3.4.2 |
| Suicide and self-harm — multi-turn evaluations — Appropriate response rate | Claude Sonnet 4.5 | 78% (± 12%) | developer-reported (Anthropic) | 3.4.2 |
| Political even-handedness (open-source Anthropic evaluation) — Even-handedness | Claude Sonnet 4.6 | 98.4% | 1,350 prompt pairs, 9 task types, 150 topics; Claude Sonnet 4.5 grader; public system prompt, standard (no) thinking; developer-reported | 3.5.1 |
| Political even-handedness (open-source Anthropic evaluation) — Even-handedness | Claude Opus 4.6 | 98.2% | developer-reported (Anthropic) | 3.5.1 |
| Political even-handedness (open-source Anthropic evaluation) — Even-handedness | Claude Opus 4.5 | 96.2% | developer-reported (Anthropic) | 3.5.1 |
| Political even-handedness (open-source Anthropic evaluation) — Even-handedness | Claude Haiku 4.5 | 88.4% | developer-reported (Anthropic) | 3.5.1 |
| Political even-handedness (open-source Anthropic evaluation) — Even-handedness | Claude Sonnet 4.5 | 94.2% | developer-reported (Anthropic) | 3.5.1 |
| Political even-handedness (open-source Anthropic evaluation) — Opposing perspectives | Claude Sonnet 4.6 | 32.1% | higher is better; developer-reported | 3.5.1 |
| Political even-handedness (open-source Anthropic evaluation) — Opposing perspectives | Claude Opus 4.6 | 44.6% | developer-reported (Anthropic) | 3.5.1 |
| Political even-handedness (open-source Anthropic evaluation) — Opposing perspectives | Claude Opus 4.5 | 40.5% | developer-reported (Anthropic) | 3.5.1 |
| Political even-handedness (open-source Anthropic evaluation) — Opposing perspectives | Claude Haiku 4.5 | 43.6% | developer-reported (Anthropic) | 3.5.1 |
| Political even-handedness (open-source Anthropic evaluation) — Opposing perspectives | Claude Sonnet 4.5 | 26.2% | developer-reported (Anthropic) | 3.5.1 |
| Political even-handedness (open-source Anthropic evaluation) — Refusals | Claude Sonnet 4.6 | 4.5% | lower is better; developer-reported | 3.5.1 |
| Political even-handedness (open-source Anthropic evaluation) — Refusals | Claude Opus 4.6 | 4.5% | developer-reported (Anthropic) | 3.5.1 |
| Political even-handedness (open-source Anthropic evaluation) — Refusals | Claude Opus 4.5 | 3.9% | developer-reported (Anthropic) | 3.5.1 |
| Political even-handedness (open-source Anthropic evaluation) — Refusals | Claude Haiku 4.5 | 4.4% | developer-reported (Anthropic) | 3.5.1 |
| Political even-handedness (open-source Anthropic evaluation) — Refusals | Claude Sonnet 4.5 | 2.2% | developer-reported (Anthropic) | 3.5.1 |
| BBQ (Bias Benchmark for Question Answering) — Disambiguated accuracy | Claude Sonnet 4.6 | 88.1 | accuracy in %, standard thinking; developer-reported | 3.5.2 |
| BBQ (Bias Benchmark for Question Answering) — Ambiguous accuracy | Claude Sonnet 4.6 | 97.5 | accuracy in %; among the 2.5% incorrect answers, 78% were stereotypical and 22% anti-stereotypical; developer-reported | 3.5.2 |
| BBQ (Bias Benchmark for Question Answering) — Disambiguated accuracy | Claude Opus 4.6 | 90.9 | developer-reported (Anthropic) | 3.5.2 |
| BBQ (Bias Benchmark for Question Answering) — Ambiguous accuracy | Claude Opus 4.6 | 99.7 | developer-reported (Anthropic) | 3.5.2 |
| BBQ (Bias Benchmark for Question Answering) — Disambiguated accuracy | Claude Opus 4.5 | 88.7 | developer-reported (Anthropic) | 3.5.2 |
| BBQ (Bias Benchmark for Question Answering) — Ambiguous accuracy | Claude Opus 4.5 | 99.7 | developer-reported (Anthropic) | 3.5.2 |
| BBQ (Bias Benchmark for Question Answering) — Disambiguated accuracy | Claude Haiku 4.5 | 71.2 | developer-reported (Anthropic) | 3.5.2 |
| BBQ (Bias Benchmark for Question Answering) — Ambiguous accuracy | Claude Haiku 4.5 | 98.0 | developer-reported (Anthropic) | 3.5.2 |
| BBQ (Bias Benchmark for Question Answering) — Disambiguated accuracy | Claude Sonnet 4.5 | 82.2 | developer-reported (Anthropic) | 3.5.2 |
| BBQ (Bias Benchmark for Question Answering) — Ambiguous accuracy | Claude Sonnet 4.5 | 99.7 | developer-reported (Anthropic) | 3.5.2 |
| BBQ (Bias Benchmark for Question Answering) — Disambiguated bias | Claude Sonnet 4.6 | -0.67 | bias score in %, closer to zero is better; standard thinking; developer-reported | 3.5.2 |
| BBQ (Bias Benchmark for Question Answering) — Ambiguous bias | Claude Sonnet 4.6 | 1.41 | developer-reported | 3.5.2 |
| BBQ (Bias Benchmark for Question Answering) — Disambiguated bias | Claude Opus 4.6 | -0.73 | developer-reported (Anthropic) | 3.5.2 |
| BBQ (Bias Benchmark for Question Answering) — Ambiguous bias | Claude Opus 4.6 | 0.14 | developer-reported (Anthropic) | 3.5.2 |
| BBQ (Bias Benchmark for Question Answering) — Disambiguated bias | Claude Opus 4.5 | -0.64 | developer-reported (Anthropic) | 3.5.2 |
| BBQ (Bias Benchmark for Question Answering) — Ambiguous bias | Claude Opus 4.5 | 0.26 | developer-reported (Anthropic) | 3.5.2 |
| BBQ (Bias Benchmark for Question Answering) — Disambiguated bias | Claude Haiku 4.5 | 0.54 | developer-reported (Anthropic) | 3.5.2 |
| BBQ (Bias Benchmark for Question Answering) — Ambiguous bias | Claude Haiku 4.5 | 1.37 | developer-reported (Anthropic) | 3.5.2 |
| BBQ (Bias Benchmark for Question Answering) — Disambiguated bias | Claude Sonnet 4.5 | -2.21 | developer-reported (Anthropic) | 3.5.2 |
| BBQ (Bias Benchmark for Question Answering) — Ambiguous bias | Claude Sonnet 4.5 | 0.25 | developer-reported (Anthropic) | 3.5.2 |
| Reward-hack-prone coding tasks — Classifier hack rate | Claude Sonnet 4.6 | 0% | blatant reward-hacking evaluations in agentic coding; developer-reported | 4.3.2 |
| Reward-hack-prone coding tasks — Hidden test hack rate | Claude Sonnet 4.6 | 0% | developer-reported | 4.3.2 |
| Impossible Tasks — Classifier hack rate with no prompt | Claude Sonnet 4.6 | 40% | developer-reported | 4.3.2 |
| Impossible Tasks — Classifier hack rate with anti-hack prompt | Claude Sonnet 4.6 | 28% | developer-reported | 4.3.2 |
| Reward-hack-prone coding tasks — Classifier hack rate | Claude Opus 4.6 | 0% | developer-reported (Anthropic) | 4.3.2 |
| Reward-hack-prone coding tasks — Hidden test hack rate | Claude Opus 4.6 | 0% | developer-reported (Anthropic) | 4.3.2 |
| Impossible Tasks — Classifier hack rate with no prompt | Claude Opus 4.6 | 50% | developer-reported (Anthropic) | 4.3.2 |
| Impossible Tasks — Classifier hack rate with anti-hack prompt | Claude Opus 4.6 | 23% | developer-reported (Anthropic) | 4.3.2 |
| Reward-hack-prone coding tasks — Classifier hack rate | Claude Opus 4.5 | 0% | developer-reported (Anthropic) | 4.3.2 |
| Reward-hack-prone coding tasks — Hidden test hack rate | Claude Opus 4.5 | 0% | developer-reported (Anthropic) | 4.3.2 |
| Impossible Tasks — Classifier hack rate with no prompt | Claude Opus 4.5 | 55% | developer-reported (Anthropic) | 4.3.2 |
| Impossible Tasks — Classifier hack rate with anti-hack prompt | Claude Opus 4.5 | 35% | developer-reported (Anthropic) | 4.3.2 |
| Reward-hack-prone coding tasks — Classifier hack rate | Claude Haiku 4.5 | 6% | developer-reported (Anthropic) | 4.3.2 |
| Reward-hack-prone coding tasks — Hidden test hack rate | Claude Haiku 4.5 | 3% | developer-reported (Anthropic) | 4.3.2 |
| Impossible Tasks — Classifier hack rate with no prompt | Claude Haiku 4.5 | 30% | developer-reported (Anthropic) | 4.3.2 |
| Impossible Tasks — Classifier hack rate with anti-hack prompt | Claude Haiku 4.5 | 23% | developer-reported (Anthropic) | 4.3.2 |
| Reward-hack-prone coding tasks — Classifier hack rate | Claude Sonnet 4.5 | 1% | developer-reported (Anthropic) | 4.3.2 |
| Reward-hack-prone coding tasks — Hidden test hack rate | Claude Sonnet 4.5 | 1% | developer-reported (Anthropic) | 4.3.2 |
| Impossible Tasks — Classifier hack rate with no prompt | Claude Sonnet 4.5 | 53% | developer-reported (Anthropic) | 4.3.2 |
| Impossible Tasks — Classifier hack rate with anti-hack prompt | Claude Sonnet 4.5 | 20% | developer-reported (Anthropic) | 4.3.2 |
| Participation in junk science — Enablement rate | Claude Sonnet 4.6 | 0 out of 90 scenarios | targeted evaluation for cooperation with generating or submitting unsound scientific content; developer-reported | 4.6.5 |
| Participation in junk science — Enablement rate | Claude Opus 4.6 | 1 out of 90 scenarios | developer-reported (Anthropic) | 4.6.5 |
| Sandbagging assessment — Model-graded transcript review | Claude Sonnet 4.6 | no instances found | random sample of 1,000 dangerous-capability evaluation transcripts from two partially-trained snapshots, the final snapshot and a helpful-only snapshot; graded by Claude Sonnet 4.5, with manual review of all nonzero sandbagging or ≥3/10 concerning scores | 4.6.4 |
| Sandbagging assessment — score regressions over training — RSP evaluations | Claude Sonnet 4.6 | 5 cases out of around 75 tasks | tasks where an earlier snapshot's 30th-percentile score exceeded the 70th-percentile score of any later snapshot, across usually 30 resampled trials; all five attributed to prompting or scaffolding changes | 4.6.4 |
| Agentic coding malicious-use evaluation (refusal rate) — No mitigations | Claude Sonnet 4.6 | 100% | 150 malicious coding requests, same coding tools as capability evaluations, no additional safeguards; developer-reported | 5.1.1 |
| Agentic coding malicious-use evaluation (refusal rate) — No mitigations | Claude Opus 4.6 | 99.3% | developer-reported (Anthropic) | 5.1.1 |
| Agentic coding malicious-use evaluation (refusal rate) — No mitigations | Claude Opus 4.5 | 100% | developer-reported (Anthropic) | 5.1.1 |
| Agentic coding malicious-use evaluation (refusal rate) — No mitigations | Claude Haiku 4.5 | 100% | developer-reported (Anthropic) | 5.1.1 |
| Agentic coding malicious-use evaluation (refusal rate) — No mitigations | Claude Sonnet 4.5 | 98.7% | developer-reported (Anthropic) | 5.1.1 |
| Claude Code malicious-use evaluation (refusal rate) — No mitigations | Claude Sonnet 4.6 | 79.34% | standard Claude Code tools, each prompt run multiple times, no mitigations; developer-reported | 5.1.2 |
| Claude Code malicious-use evaluation (refusal rate) — No mitigations | Claude Opus 4.6 | 83.20% | developer-reported (Anthropic) | 5.1.2 |
| Claude Code malicious-use evaluation (refusal rate) — No mitigations | Claude Opus 4.5 | 77.80% | developer-reported (Anthropic) | 5.1.2 |
| Claude Code malicious-use evaluation (refusal rate) — No mitigations | Claude Haiku 4.5 | 69.39% | developer-reported (Anthropic) | 5.1.2 |
| Claude Code malicious-use evaluation (refusal rate) — No mitigations | Claude Sonnet 4.5 | 63.06% | developer-reported (Anthropic) | 5.1.2 |
| Claude Code dual-use & benign evaluation (success rate) — No mitigations | Claude Sonnet 4.6 | 88.52% | developer-reported | 5.1.2 |
| Claude Code dual-use & benign evaluation (success rate) — No mitigations | Claude Opus 4.6 | 91.75% | developer-reported (Anthropic) | 5.1.2 |
| Claude Code dual-use & benign evaluation (success rate) — No mitigations | Claude Opus 4.5 | 93.07% | developer-reported (Anthropic) | 5.1.2 |
| Claude Code dual-use & benign evaluation (success rate) — No mitigations | Claude Haiku 4.5 | 84.92% | developer-reported (Anthropic) | 5.1.2 |
| Claude Code dual-use & benign evaluation (success rate) — No mitigations | Claude Sonnet 4.5 | 96.56% | developer-reported (Anthropic) | 5.1.2 |
| Claude Code malicious-use evaluation (refusal rate) — With mitigations | Claude Sonnet 4.6 | 99.39% | Claude Code system prompt plus a FileRead tool-result reminder; developer-reported | 5.1.2 |
| Claude Code malicious-use evaluation (refusal rate) — With mitigations | Claude Opus 4.6 | 99.59% | developer-reported (Anthropic) | 5.1.2 |
| Claude Code malicious-use evaluation (refusal rate) — With mitigations | Claude Opus 4.5 | 97.35% | developer-reported (Anthropic) | 5.1.2 |
| Claude Code malicious-use evaluation (refusal rate) — With mitigations | Claude Haiku 4.5 | 96.73% | developer-reported (Anthropic) | 5.1.2 |
| Claude Code malicious-use evaluation (refusal rate) — With mitigations | Claude Sonnet 4.5 | 95.10% | developer-reported (Anthropic) | 5.1.2 |
| Claude Code dual-use & benign evaluation (success rate) — With mitigations | Claude Sonnet 4.6 | 91.78% | developer-reported | 5.1.2 |
| Claude Code dual-use & benign evaluation (success rate) — With mitigations | Claude Opus 4.6 | 95.59% | developer-reported (Anthropic) | 5.1.2 |
| Claude Code dual-use & benign evaluation (success rate) — With mitigations | Claude Opus 4.5 | 96.52% | developer-reported (Anthropic) | 5.1.2 |
| Claude Code dual-use & benign evaluation (success rate) — With mitigations | Claude Haiku 4.5 | 86.07% | developer-reported (Anthropic) | 5.1.2 |
| Claude Code dual-use & benign evaluation (success rate) — With mitigations | Claude Sonnet 4.5 | 98.20% | developer-reported (Anthropic) | 5.1.2 |
| Malicious computer use evaluation (refusal rate) — No mitigations | Claude Sonnet 4.6 | 99.38% | 112 tasks run under both extended and standard thinking, 224 attempts, sandboxed GUI/CLI environment; developer-reported | 5.1.3 |
| Malicious computer use evaluation (refusal rate) — No mitigations | Claude Opus 4.6 | 88.34% | developer-reported (Anthropic) | 5.1.3 |
| Malicious computer use evaluation (refusal rate) — No mitigations | Claude Opus 4.5 | 88.39% | developer-reported (Anthropic) | 5.1.3 |
| Malicious computer use evaluation (refusal rate) — No mitigations | Claude Haiku 4.5 | 77.68% | developer-reported (Anthropic); the Haiku 4.5 and Sonnet 4.5 values differ from those in the Opus 4.5 card | 5.1.3 |
| Malicious computer use evaluation (refusal rate) — No mitigations | Claude Sonnet 4.5 | 86.08% | developer-reported (Anthropic) | 5.1.3 |
| Shade adaptive indirect prompt injection (coding) — 1 attempt, extended thinking, without safeguards | Claude Sonnet 4.6 | 0.0% | attack success rate, lower is better; Gray Swan Shade adaptive red-teaming tool | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 200 attempts, extended thinking, without safeguards | Claude Sonnet 4.6 | 0.0% | ASR; at least one of 200 attempts succeeding for a given goal | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 1 attempt, extended thinking, with safeguards | Claude Sonnet 4.6 | 0.0% | ASR | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 200 attempts, extended thinking, with safeguards | Claude Sonnet 4.6 | 0.0% | ASR | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 1 attempt, standard thinking, without safeguards | Claude Sonnet 4.6 | 0.1% | ASR | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 200 attempts, standard thinking, without safeguards | Claude Sonnet 4.6 | 7.5% | ASR | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 1 attempt, standard thinking, with safeguards | Claude Sonnet 4.6 | 0.04% | ASR | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 200 attempts, standard thinking, with safeguards | Claude Sonnet 4.6 | 5.0% | ASR | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 1 attempt, extended thinking, without safeguards | Claude Opus 4.6 | 0.0% | ASR | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 200 attempts, extended thinking, without safeguards | Claude Opus 4.6 | 0.0% | ASR | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 1 attempt, standard thinking, without safeguards | Claude Opus 4.6 | 0.0% | ASR | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 200 attempts, standard thinking, without safeguards | Claude Opus 4.6 | 0.0% | ASR | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 1 attempt, extended thinking, with safeguards | Claude Opus 4.6 | 0.0% | ASR | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 200 attempts, extended thinking, with safeguards | Claude Opus 4.6 | 0.0% | ASR | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 1 attempt, standard thinking, with safeguards | Claude Opus 4.6 | 0.0% | ASR | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 200 attempts, standard thinking, with safeguards | Claude Opus 4.6 | 0.0% | ASR | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 1 attempt, extended thinking, without safeguards | Claude Opus 4.5 | 0.3% | ASR | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 200 attempts, extended thinking, without safeguards | Claude Opus 4.5 | 10.0% | ASR | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 1 attempt, extended thinking, with safeguards | Claude Opus 4.5 | 0.1% | ASR | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 200 attempts, extended thinking, with safeguards | Claude Opus 4.5 | 7.5% | ASR | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 1 attempt, standard thinking, without safeguards | Claude Opus 4.5 | 0.7% | ASR | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 200 attempts, standard thinking, without safeguards | Claude Opus 4.5 | 17.5% | ASR | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 1 attempt, standard thinking, with safeguards | Claude Opus 4.5 | 0.2% | ASR | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 200 attempts, standard thinking, with safeguards | Claude Opus 4.5 | 7.5% | ASR | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 1 attempt, extended thinking, without safeguards | Claude Sonnet 4.5 | 18.3% | ASR; differs from the 17.7% reported in the Opus 4.5 card | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 200 attempts, extended thinking, without safeguards | Claude Sonnet 4.5 | 70.0% | ASR | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 1 attempt, extended thinking, with safeguards | Claude Sonnet 4.5 | 1.6% | ASR | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 200 attempts, extended thinking, with safeguards | Claude Sonnet 4.5 | 25.0% | ASR | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 1 attempt, standard thinking, without safeguards | Claude Sonnet 4.5 | 31.6% | ASR; differs from the 29.9% reported in the Opus 4.5 card | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 200 attempts, standard thinking, without safeguards | Claude Sonnet 4.5 | 87.5% | ASR | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 1 attempt, standard thinking, with safeguards | Claude Sonnet 4.5 | 1.7% | ASR | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 200 attempts, standard thinking, with safeguards | Claude Sonnet 4.5 | 25.0% | ASR | 5.2.2.1 |
| Shade adaptive indirect prompt injection (computer use) — 1 attempt, extended thinking, without safeguards | Claude Sonnet 4.6 | 12.0% | ASR in GUI computer-use environments | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 200 attempts, extended thinking, without safeguards | Claude Sonnet 4.6 | 42.9% | ASR | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 1 attempt, extended thinking, with safeguards | Claude Sonnet 4.6 | 8.0% | ASR | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 200 attempts, extended thinking, with safeguards | Claude Sonnet 4.6 | 50.0% | ASR | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 1 attempt, standard thinking, without safeguards | Claude Sonnet 4.6 | 14.4% | ASR | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 200 attempts, standard thinking, without safeguards | Claude Sonnet 4.6 | 64.3% | ASR | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 1 attempt, standard thinking, with safeguards | Claude Sonnet 4.6 | 8.6% | ASR | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 200 attempts, standard thinking, with safeguards | Claude Sonnet 4.6 | 50.0% | ASR | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 1 attempt, extended thinking, without safeguards | Claude Opus 4.6 | 17.8% | ASR | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 200 attempts, extended thinking, without safeguards | Claude Opus 4.6 | 78.6% | ASR | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 1 attempt, extended thinking, with safeguards | Claude Opus 4.6 | 9.7% | ASR | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 200 attempts, extended thinking, with safeguards | Claude Opus 4.6 | 57.1% | ASR | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 1 attempt, standard thinking, without safeguards | Claude Opus 4.6 | 20.0% | ASR | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 200 attempts, standard thinking, without safeguards | Claude Opus 4.6 | 85.7% | ASR | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 1 attempt, standard thinking, with safeguards | Claude Opus 4.6 | 10.0% | ASR | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 200 attempts, standard thinking, with safeguards | Claude Opus 4.6 | 64.3% | ASR | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 1 attempt, extended thinking, without safeguards | Claude Opus 4.5 | 28.0% | ASR; differs from the 0.0% reported in the Opus 4.5 card for extended thinking | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 200 attempts, extended thinking, without safeguards | Claude Opus 4.5 | 78.6% | ASR | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 1 attempt, extended thinking, with safeguards | Claude Opus 4.5 | 17.3% | ASR | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 200 attempts, extended thinking, with safeguards | Claude Opus 4.5 | 64.3% | ASR | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 1 attempt, standard thinking, without safeguards | Claude Opus 4.5 | 35.4% | ASR | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 200 attempts, standard thinking, without safeguards | Claude Opus 4.5 | 85.7% | ASR | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 1 attempt, standard thinking, with safeguards | Claude Opus 4.5 | 18.8% | ASR | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 200 attempts, standard thinking, with safeguards | Claude Opus 4.5 | 71.4% | ASR | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 1 attempt, extended thinking, without safeguards | Claude Sonnet 4.5 | 41.8% | ASR | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 200 attempts, extended thinking, without safeguards | Claude Sonnet 4.5 | 92.9% | ASR | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 1 attempt, extended thinking, with safeguards | Claude Sonnet 4.5 | 25.2% | ASR | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 200 attempts, extended thinking, with safeguards | Claude Sonnet 4.5 | 85.7% | ASR | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 1 attempt, standard thinking, without safeguards | Claude Sonnet 4.5 | 19.0% | ASR | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 200 attempts, standard thinking, without safeguards | Claude Sonnet 4.5 | 92.9% | ASR | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 1 attempt, standard thinking, with safeguards | Claude Sonnet 4.5 | 12.8% | ASR | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 200 attempts, standard thinking, with safeguards | Claude Sonnet 4.5 | 71.4% | ASR | 5.2.2.2 |
| Internal browser-use Best-of-N prompt injection (no safeguards) — % of scenarios, extended thinking | Claude Sonnet 4.6 | 1.29% | 10 attack strings across 389 scenarios; per-environment ASR (at least one attempt succeeded) | 5.2.2.3 |
| Internal browser-use Best-of-N prompt injection (no safeguards) — % of attempts, extended thinking | Claude Sonnet 4.6 | 0.24% | per-attempt ASR | 5.2.2.3 |
| Internal browser-use Best-of-N prompt injection (no safeguards) — % of scenarios, standard thinking | Claude Sonnet 4.6 | 1.29% | per-environment ASR | 5.2.2.3 |
| Internal browser-use Best-of-N prompt injection (no safeguards) — % of attempts, standard thinking | Claude Sonnet 4.6 | 0.29% | per-attempt ASR | 5.2.2.3 |
| Internal browser-use Best-of-N prompt injection (no safeguards) — % of scenarios, extended thinking | Claude Opus 4.6 | 2.06% | per-environment ASR | 5.2.2.3 |
| Internal browser-use Best-of-N prompt injection (no safeguards) — % of attempts, extended thinking | Claude Opus 4.6 | 0.29% | per-attempt ASR | 5.2.2.3 |
| Internal browser-use Best-of-N prompt injection (no safeguards) — % of scenarios, standard thinking | Claude Opus 4.6 | 2.83% | per-environment ASR | 5.2.2.3 |
| Internal browser-use Best-of-N prompt injection (no safeguards) — % of attempts, standard thinking | Claude Opus 4.6 | 0.49% | per-attempt ASR | 5.2.2.3 |
| Internal browser-use Best-of-N prompt injection (no safeguards) — % of scenarios, extended thinking | Claude Opus 4.5 | 18.77% | per-environment ASR | 5.2.2.3 |
| Internal browser-use Best-of-N prompt injection (no safeguards) — % of attempts, extended thinking | Claude Opus 4.5 | 6.40% | per-attempt ASR | 5.2.2.3 |
| Internal browser-use Best-of-N prompt injection (no safeguards) — % of scenarios, standard thinking | Claude Opus 4.5 | 16.20% | per-environment ASR | 5.2.2.3 |
| Internal browser-use Best-of-N prompt injection (no safeguards) — % of attempts, standard thinking | Claude Opus 4.5 | 5.06% | per-attempt ASR | 5.2.2.3 |
| Internal browser-use Best-of-N prompt injection (no safeguards) — % of scenarios, extended thinking | Claude Sonnet 4.5 | 54.24% | per-environment ASR | 5.2.2.3 |
| Internal browser-use Best-of-N prompt injection (no safeguards) — % of attempts, extended thinking | Claude Sonnet 4.5 | 20.45% | per-attempt ASR | 5.2.2.3 |
| Internal browser-use Best-of-N prompt injection (no safeguards) — % of scenarios, standard thinking | Claude Sonnet 4.5 | 49.36% | per-environment ASR | 5.2.2.3 |
| Internal browser-use Best-of-N prompt injection (no safeguards) — % of attempts, standard thinking | Claude Sonnet 4.5 | 16.23% | per-attempt ASR | 5.2.2.3 |
| Internal browser-use Best-of-N prompt injection (previous safeguards) — % of scenarios | Claude Sonnet 4.6 | 1.03% | standard thinking, previous safeguards | 5.2.2.3 |
| Internal browser-use Best-of-N prompt injection (previous safeguards) — % of attempts | Claude Sonnet 4.6 | 0.16% | standard thinking, previous safeguards | 5.2.2.3 |
| Internal browser-use Best-of-N prompt injection (updated safeguards) — % of scenarios | Claude Sonnet 4.6 | 0.51% | standard thinking, updated safeguards | 5.2.2.3 |
| Internal browser-use Best-of-N prompt injection (updated safeguards) — % of attempts | Claude Sonnet 4.6 | 0.08% | standard thinking, updated safeguards | 5.2.2.3 |
| Internal browser-use Best-of-N prompt injection (previous safeguards) — % of scenarios | Claude Opus 4.6 | 0.26% | standard thinking | 5.2.2.3 |
| Internal browser-use Best-of-N prompt injection (previous safeguards) — % of attempts | Claude Opus 4.6 | 0.03% | standard thinking | 5.2.2.3 |
| Internal browser-use Best-of-N prompt injection (updated safeguards) — % of scenarios | Claude Opus 4.6 | 0.77% | standard thinking | 5.2.2.3 |
| Internal browser-use Best-of-N prompt injection (updated safeguards) — % of attempts | Claude Opus 4.6 | 0.08% | standard thinking | 5.2.2.3 |
| Internal browser-use Best-of-N prompt injection (previous safeguards) — % of scenarios | Claude Opus 4.5 | 1.03% | standard thinking | 5.2.2.3 |
| Internal browser-use Best-of-N prompt injection (previous safeguards) — % of attempts | Claude Opus 4.5 | 0.21% | standard thinking | 5.2.2.3 |
| Internal browser-use Best-of-N prompt injection (updated safeguards) — % of scenarios | Claude Opus 4.5 | 1.54% | standard thinking | 5.2.2.3 |
| Internal browser-use Best-of-N prompt injection (updated safeguards) — % of attempts | Claude Opus 4.5 | 0.40% | standard thinking | 5.2.2.3 |
| Internal browser-use Best-of-N prompt injection (previous safeguards) — % of scenarios | Claude Sonnet 4.5 | 1.54% | standard thinking | 5.2.2.3 |
| Internal browser-use Best-of-N prompt injection (previous safeguards) — % of attempts | Claude Sonnet 4.5 | 0.41% | standard thinking | 5.2.2.3 |
| Internal browser-use Best-of-N prompt injection (updated safeguards) — % of scenarios | Claude Sonnet 4.5 | 2.06% | standard thinking | 5.2.2.3 |
| Internal browser-use Best-of-N prompt injection (updated safeguards) — % of attempts | Claude Sonnet 4.5 | 0.46% | standard thinking | 5.2.2.3 |
| Long-form virology task 1 — Max of 5 | Claude Sonnet 4.6 | 0.84 | agentic pathogen-acquisition workflow task with SecureBio, Deloitte and Signature Science; ASL-3 rule-in threshold 0.8, rule-out <50%; highest-scoring snapshot reported; developer-reported | 6.2.2.1 |
| Long-form virology task 1 — Max of 5 | Claude Opus 4.6 | 0.79 | developer-reported (Anthropic) | 6.2.2.1 |
| Long-form virology task 1 — Max of 5 | Claude Opus 4.5 | 0.85 | developer-reported (Anthropic); the Opus 4.5 card reported 0.845 | 6.2.2.1 |
| Long-form virology task 1 — Max of 5 | Claude Sonnet 4.5 | 0.89 | developer-reported (Anthropic) | 6.2.2.1 |
| Long-form virology task 2 — Max of 5 | Claude Sonnet 4.6 | 0.89 | exceeds the 0.8 threshold; developer-reported | 6.2.2.1 |
| Long-form virology task 2 — Max of 5 | Claude Opus 4.6 | 0.91 | developer-reported (Anthropic) | 6.2.2.1 |
| Long-form virology task 2 — Max of 5 | Claude Opus 4.5 | 0.91 | developer-reported (Anthropic); the Opus 4.5 card reported 0.912 as a "pass@5" score | 6.2.2.1 |
| DNA Synthesis Screening Evasion — Plasmids succeeded | Claude Sonnet 4.6 | 3/10 plasmids | 10 pathogenic agents; low-concern threshold is viable plasmids evading screening for all 10 pathogens on at least one method; developer-reported | 6.2.2.3 |
| DNA Synthesis Screening Evasion — Plasmids succeeded | Claude Opus 4.5 | 4/10 | developer-reported (Anthropic) | 6.2.2.3 |
| DNA Synthesis Screening Evasion — Plasmids succeeded | Claude Opus 4.6 | 5/10 | developer-reported (Anthropic) | 6.2.2.3 |
| Creative biology (SecureBio) | Claude Sonnet 4.6 | 0.593 | developer-reported | 6.2.2.4 |
| Creative biology (SecureBio) | Claude Opus 4.6 | 0.603 | developer-reported (Anthropic) | 6.2.2.4 |
| Creative biology (SecureBio) | Claude Opus 4.5 | 0.524 | developer-reported (Anthropic) | 6.2.2.4 |
| Creative biology (SecureBio) | Claude Sonnet 4.5 | 0.506 | developer-reported (Anthropic); the Opus 4.5 card reported 0.488 for Sonnet 4.5 | 6.2.2.4 |
| Creative biology (SecureBio) | Human baseline (biology PhDs, subset) | 14% | preliminary SecureBio human baselining on a subset of questions | 6.2.2.4 |
| Short-horizon computational biology tasks (Faculty.ai) — Lower-bound rule-out thresholds crossed | Claude Sonnet 4.6 | 3/6 | scored similarly to Opus 4.5 on 5 of 6 tasks and similar to Opus 4.6 on 1 of 6 (task 5); developer-reported | 6.2.2.5 |
| SWE-bench Verified (hard subset) — Verified (hard subset, 45 tasks) | Claude Sonnet 4.6 | 21.7 out of 45 problems | average over 10 trials; RSP checkpoint threshold is a >50% pass rate, i.e. 22.5 problems solved on average; developer-reported | 6.3.2 |
| Internal AI research evaluation suite 1 — Kernels task — Best speedup | Claude Sonnet 4.6 | 222.5× | standard scaffold; threshold 100×; 4× ≈ 1 human-effort hour, 200× ≈ 8 hours, 300× ≈ 40 hours; developer-reported | 6.3.3.1 |
| Internal AI research evaluation suite 1 — Time series forecasting — Hard variant | Claude Sonnet 4.6 | 5.70 | minimum MSE, above (worse than) the 5.3 human baseline; two sub-threshold trials were invalidated on transcript review for using test ground-truth data; developer-reported | 6.3.3.2 |
| Internal AI research evaluation suite 1 — Time series forecasting — Hard variant | Human baseline | 5.3 | MSE human baseline for the hard variant | 6.3.3.2 |
| Internal AI research evaluation suite 1 — Text-based reinforcement learning task — Best score | Claude Sonnet 4.6 | 0.925 | above the 0.9 threshold (≈8 human-effort hours); average performance lower than previous models including Sonnet 4.5; developer-reported | 6.3.3.3 |
| Internal AI research evaluation suite 1 — LLM training — Average best speedup | Claude Sonnet 4.6 | 16.53× | threshold 4× speedup (4–8 human-effort hours); developer-reported | 6.3.3.4 |
| Internal AI research evaluation suite 1 — Quadruped reinforcement learning — No-hyperparameter variant | Claude Sonnet 4.6 | 18.72 | highest score; threshold 12 (≈4 human-effort hours); developer-reported | 6.3.3.5 |
| Internal AI research evaluation suite 1 — Quadruped reinforcement learning — No-reward-function variant | Claude Sonnet 4.6 | 17.88 | highest score; threshold 12; developer-reported | 6.3.3.5 |
| Internal AI research evaluation suite 1 — Novel compiler — Basic tests | Claude Sonnet 4.6 | 93.7% | share of basic test cases passed; success threshold 90%; developer-reported | 6.3.3.6 |
| Internal AI research evaluation suite 1 — Novel compiler — Complex tests | Claude Sonnet 4.6 | 67.27% | share of complex test cases passed, below the 90% threshold (≈40 human-effort hours); developer-reported | 6.3.3.6 |
| RSP cyber CTFs — Web — Challenges solved | Claude Sonnet 4.6 | 13 out of 13 total | figure caption; section text describes 15 challenges (11 easy, 2 medium, 2 hard); developer-reported | 6.4.2 |
| RSP cyber CTFs — Crypto — Challenges solved | Claude Sonnet 4.6 | 16 out of 18 total | figure caption; section text describes 22 challenges (7 easy, 6 medium, 9 hard); developer-reported | 6.4.3 |
| RSP cyber CTFs — Pwn — Challenges solved | Claude Sonnet 4.6 | 5 out of 7 total | figure caption; section text describes 9 challenges; developer-reported | 6.4.4 |
| RSP cyber CTFs — Rev — Challenges solved | Claude Sonnet 4.6 | 6 out of 6 total | figure caption; section text describes 8 challenges; developer-reported | 6.4.5 |
| RSP cyber CTFs — Network — Challenges solved | Claude Sonnet 4.6 | 5 out of 5 total | figure caption; section text describes 4 challenges; developer-reported | 6.4.6 |
| OpenRCA — Overall | Claude Sonnet 4.6 | 27.9% | adaptive thinking, high effort; 335 failure cases, benchmark authors' agent harness; 3-run average; developer-reported | 2.4 |
| OpenRCA — Overall | Claude Sonnet 4.6 | 26.4% | adaptive thinking, max effort; 3-run average; developer-reported | 2.4 |
| OpenRCA — Overall | Claude Opus 4.6 | 34.9% | thinking enabled, default effort; 3-run average; developer-reported (Anthropic) | 2.4 |
| OpenRCA — Overall | Claude Opus 4.5 | 26.9% | 3-run average; developer-reported (Anthropic) | 2.4 |
| OpenRCA — Overall | Claude Sonnet 4.5 | 12.9% | 3-run average; developer-reported (Anthropic) | 2.4 |
| OpenRCA — Overall | GPT-5.2 | 19.4% | reported by the benchmark authors, per Anthropic | 2.4 |
| OpenRCA — Overall | Gemini 3 Pro | 12.5% | reported by the benchmark authors, per Anthropic | 2.4 |
| CyberGym | Claude Sonnet 4.6 | 65.2% | pass@1 on targeted vulnerability reproduction over the 1,507 tasks of the CyberGym suite; no thinking, default effort/temperature/top_p, "think" tool available; developer-reported | 2.15 |
| CyberGym | Claude Opus 4.6 | 66.6% | developer-reported (Anthropic) | 2.15 |
| CyberGym | Claude Opus 4.5 | 51.0% | developer-reported (Anthropic) | 2.15 |
| CyberGym | Claude Sonnet 4.5 | 29.8% | developer-reported (Anthropic) | 2.15 |
| GraphWalks BFS 1M | Claude Sonnet 4.6 | 68.4 | F1; 64k extended thinking; average over 5 trials with 1M context, default sampling; 100 problems at 1024k context; not reproducible via the public API; developer-reported | 2.16 |
| GraphWalks BFS 1M | Claude Sonnet 4.6 | 73.8 | F1; max effort; developer-reported | 2.16 |
| GraphWalks BFS 1M | Claude Opus 4.6 | 41.2 | F1; 64k extended thinking; developer-reported (Anthropic) | 2.16 |
| GraphWalks BFS 1M | Claude Opus 4.6 | 38.7 | F1; max effort; developer-reported (Anthropic) | 2.16 |
| GraphWalks BFS 1M | Claude Sonnet 4.5 | 25.6 | F1; 64k extended thinking; developer-reported (Anthropic) | 2.16 |
| GraphWalks BFS 256K subset of 1M | Claude Sonnet 4.6 | 72.8 | F1; 64k extended thinking; subset reproducible under the 1M API token limit; developer-reported | 2.16 |
| GraphWalks BFS 256K subset of 1M | Claude Sonnet 4.6 | 74.5 | F1; max effort; developer-reported | 2.16 |
| GraphWalks BFS 256K subset of 1M | Claude Opus 4.6 | 61.5 | F1; 64k extended thinking; developer-reported (Anthropic) | 2.16 |
| GraphWalks BFS 256K subset of 1M | Claude Opus 4.6 | 61.1 | F1; max effort; developer-reported (Anthropic) | 2.16 |
| GraphWalks BFS 256K subset of 1M | Claude Sonnet 4.5 | 44.9 | F1; 64k extended thinking; developer-reported (Anthropic) | 2.16 |
| GraphWalks Parents 256K subset of 1M | Claude Sonnet 4.6 | 96.9 | F1; 64k extended thinking; developer-reported | 2.16 |
| GraphWalks Parents 256K subset of 1M | Claude Sonnet 4.6 | 97.9 | F1; max effort; developer-reported | 2.16 |
| GraphWalks Parents 256K subset of 1M | Claude Opus 4.6 | 95.1 | F1; 64k extended thinking; developer-reported (Anthropic) | 2.16 |
| GraphWalks Parents 256K subset of 1M | Claude Opus 4.6 | 95.4 | F1; max effort; developer-reported (Anthropic) | 2.16 |
| GraphWalks Parents 256K subset of 1M | Claude Sonnet 4.5 | 81.0 | F1; 64k extended thinking; developer-reported (Anthropic) | 2.16 |
| DeepSearchQA — Single agent | Claude Sonnet 4.6 | 89.2% | F1; web search, web fetch, programmatic tool calling, context compaction at 50k tokens up to 10M total tokens, max effort, adaptive thinking; 900 prompts across 17 fields; developer-reported | 2.20.3.1 |
| DeepSearchQA — Multi-agent | Claude Sonnet 4.6 | 91.1% | F1; orchestrator and subagents each compacting at 50k tokens up to 3M tokens, max reasoning effort; developer-reported | 2.20.3.1 |
| BioPipelineBench (internal computational biology) | Claude Sonnet 4.6 | 52.1% | bash tool and package managers available, no extended thinking; developer-reported | 2.21.1 |
| BioPipelineBench (internal computational biology) | Claude Opus 4.6 | 53.1% | developer-reported (Anthropic) | 2.21.1 |
| BioPipelineBench (internal computational biology) | Claude Sonnet 4.5 | 19.3% | developer-reported (Anthropic) | 2.21.1 |
| BioMysteryBench (internal computational biology) | Claude Sonnet 4.6 | 50.4% | developer-reported | 2.21.1 |
| BioMysteryBench (internal computational biology) | Claude Opus 4.6 | 61.5% | developer-reported (Anthropic) | 2.21.1 |
| BioMysteryBench (internal computational biology) | Claude Sonnet 4.5 | 34.7% | developer-reported (Anthropic) | 2.21.1 |
| Structural Biology (internal) — Multiple-choice variant | Claude Sonnet 4.6 | 85.3% | developer-reported | 2.21.1 |
| Structural Biology (internal) — Multiple-choice variant | Claude Opus 4.6 | 88.3% | developer-reported (Anthropic) | 2.21.1 |
| Structural Biology (internal) — Multiple-choice variant | Claude Sonnet 4.5 | 70.9% | developer-reported (Anthropic) | 2.21.1 |
| Structural Biology (internal) — Open-ended variant | Claude Sonnet 4.6 | 24.7% | developer-reported | 2.21.1 |
| Structural Biology (internal) — Open-ended variant | Claude Opus 4.6 | 28.4% | developer-reported (Anthropic) | 2.21.1 |
| Structural Biology (internal) — Open-ended variant | Claude Sonnet 4.5 | 17.9% | developer-reported (Anthropic) | 2.21.1 |
| Organic Chemistry (internal) | Claude Sonnet 4.6 | 48.4% | developer-reported | 2.21.1 |
| Organic Chemistry (internal) | Claude Opus 4.6 | 53.9% | developer-reported (Anthropic) | 2.21.1 |
| Organic Chemistry (internal) | Claude Sonnet 4.5 | 31.2% | developer-reported (Anthropic) | 2.21.1 |
| Phylogenetics (internal) | Claude Sonnet 4.6 | 49.1% | developer-reported | 2.21.1 |
| Phylogenetics (internal) | Claude Opus 4.6 | 61.3% | developer-reported (Anthropic) | 2.21.1 |
| Phylogenetics (internal) | Claude Sonnet 4.5 | 33.8% | developer-reported (Anthropic) | 2.21.1 |
| MedCalc-Bench Verified | Claude Sonnet 4.6 | 86.24% | accuracy averaged over 5 runs; code-augmented methodology, multi-turn agentic loop with a Python REPL; adaptive thinking, max effort; developer-reported | 2.21.2 |
| MedCalc-Bench Verified | Claude Opus 4.6 | 85.24% | adaptive thinking, max effort; developer-reported (Anthropic) | 2.21.2 |
Discrepancies noted
Table 2.1.A's note says all results are an average over 10 trials unless otherwise noted; footnote 2 to the same table says SWE-bench results are averaged over 25 trials, and section 2.2 gives 10. The two BrowseComp figures published at launch (74.72% single-agent, 82.62% multi-agent) were revised by the card's changelog to 74.01% and 82.07%; the revised figures are recorded above.
Provenance
Published by Anthropic on February 17, 2026 (Source: anthropic.com). The raw capture runs to roughly 29,000 words across seven chapters.
Relationships
- related: Claude Sonnet 4.6 — model page for Claude Sonnet 4.6
- related: Claude Opus 4.6 System Card — companion system card for Opus 4.6, which this card repeatedly defers to for method detail
- depends-on: Anthropic's Responsible Scaling Policy (Version 3.1) — RSP v3.1 defines the ASL-3 Standard applied and the AI R&D-4 and CBRN-4 thresholds
- related: AI Safety Cases and Frameworks — documents safety framework implementation
- related: OpenRCA — benchmark introduced to Anthropic reporting in this card generation
- related: Andon Labs — third-party experiments cited in the alignment assessment
- related: Anthropic — author