AI Policy Wiki
Dashboard
⚲

Claude Sonnet 4.6 System Card

medium confidence · updated 2026-09-22

Anthropic's system card for Claude Sonnet 4.6 (February 17, 2026) — capability evaluations, safeguards and bias testing, a lighter-than-frontier alignment assessment, agentic safety, and the Responsible Scaling Policy determinations, including the proactive adoption of AI R&D-4 measures.

The Claude Sonnet 4.6 system card is Anthropic's technical and safety documentation for its mid-tier model Claude Sonnet 4.6, published February 17, 2026 alongside the model's release. It is organized into a release-decision section, capability evaluations, safeguards and bias testing, an alignment assessment, agentic safety evaluations, and evaluations tied to Anthropic's Responsible Scaling Policy, followed by an appendix. Anthropic states that it ran a similar set of evaluations to those for Claude Opus 4.6 "though with somewhat less detail since Sonnet 4.6 is not a frontier model (that is, it does not broadly advance the frontier of AI capabilities compared to the state-of-the-art in the" field). The model is deployed under the ASL-3 Standard defined in RSP v3.1 (Anthropic's Responsible Scaling Policy (Version 3.1)).

Capabilities

The card reports evaluations of software engineering, reasoning and mathematics, agentic search, computer use, knowledge work, finance, and life sciences, and — for the first time in an Anthropic card — the gap between the model's accuracy in English and in a set of low-resource languages. Anthropic notes that public benchmark data may be included in training data, making results "potentially contaminated," and refers to the Claude Opus 4.5 system card for its decontamination procedure.

Headline settings for the comparison table are an average over 10 trials (25 for SWE-bench), adaptive thinking, max effort, default sampling, and evaluation-dependent context windows never exceeding 1M tokens. Sonnet 4.6 introduces thinking modes and an effort parameter allowing the model to vary its reasoning depth, documented in section 1.1.2.

The card's individual quantitative results are recorded on the corresponding benchmark pages, which cite this card: SWE-bench, Terminal-Bench, OpenRCA, τ-bench, OSWorld, ARC-AGI, GDPval, GPQA, AIME (as a model benchmark), MMLU, Finance Agent, Vending-Bench, MCP Atlas, CyberGym, MRCR, LAB-Bench, MMMU, CharXiv, WebArena, BrowseComp, Humanity's Last Exam and Cybench.

On the headline comparison the card reports Sonnet 4.6 leading Claude Opus 4.5, Gemini 3 Pro and GPT-5.2 on several agentic measures while trailing Claude Opus 4.6 on most, and records footnoted effort-level variations in both directions: Sonnet 4.6 reaches 60.42% on ARC-AGI-2 at high effort against the 58.3% max-effort figure tabulated, while Opus 4.6 reaches 62.7% on MCP-Atlas and 69.2% on ARC-AGI-2 at high effort.

A prompt modification tested on SWE-bench Verified — instructing the model to use tools more than 100 times, write its own tests first, explore the codebase to find root causes rather than surface symptoms, and cover edge cases — raised the score from 79.6% to 80.2%.

Multilingual performance

The multilingual section is new to this card. On GMMLU the average gap to English was −4.4% for Sonnet 4.6, against −5.4% for Sonnet 4.5, −3.9% for Opus 4.6, −2.7% for Gemini 3 Pro and −3.1% for GPT-5.2 Pro. Anthropic states that degradation "is concentrated in low-resource African languages—Igbo, Chichewa, Yoruba, Shona, and Somali—a pattern consistent across Claude family models," and that a research effort to improve low-resource-language performance is underway. On MILU the average English-to-Indic gap was −2.3%, an improvement over Sonnet 4.5 (−2.8%) with gains on every evaluated language, smaller than Opus 4.6 (−2.7%) and GPT-5.2 Pro (−2.7%) but larger than Gemini 3 Pro (−2.0%).

The section also reports a measurement of test-time compute efficiency. Adaptive thinking at max effort raised Sonnet 4.6's GMMLU score by roughly 7.0 percentage points over thinking disabled; median thinking-token usage across 100 English GMMLU questions was 1,078 tokens per question for Gemini 3 Pro, 437 for Sonnet 4.5, 246 for Sonnet 4.6, 191 for Opus 4.6 and 127 for GPT-5.2 Pro, which Anthropic reads as indicating "that models can achieve comparable accuracy at very different levels of test-time compute efficiency."

Safeguards and harmlessness

Chapter 3 covers single-turn violative- and benign-request evaluations, an experimental higher-difficulty tier of each, ambiguous-context evaluations, multi-turn testing, user-wellbeing evaluations covering child safety, suicide and self-harm, and eating disorders, and bias evaluations covering political evenhandedness and the Bias Benchmark for Question Answering. Anthropic reports that overrefusal rates improved substantially from Sonnet 4.5 in adaptive multi-turn testing, though remaining above Opus 4.6.

Alignment assessment

The alignment assessment covers, in Anthropic's framing, "undesirable or hidden goals, knowingly cooperating with misuse, using reasoning scratchpads in deceptive or unfaithful ways, sycophancy toward users, willingness to sabotage our safeguards, attempts to hide dangerous capabilities, and attempts to manipulate users toward certain views." Testing ran continuously through fine-tuning, with results reported for the final model and for earlier training snapshots.

Anthropic states explicitly that the assessment was lighter than the one for Opus 4.6: it reused many of the same methods but omitted components including interpretability-augmented investigations, and "was not able to arrange for an in-depth alignment-focused third-party assessment, in part because either we or the potential assessors prioritized effort on models that advanced the capability frontier." It adds that Anthropic aims "to set a higher standard for the whole field than we meet now in our investigation of Sonnet 4.6, Opus 4.6, or any other model." The methods used include static behavioral evaluations, automated interactive behavioral evaluations, non-assistant persona sampling, misalignment-related capability evaluations, training data review, internal and external pilot-use feedback and automated analysis of it, and third-party experiments at Andon Labs. The work covered manual expert inspection of hundreds of transcripts, generation of tens of thousands of targeted evaluation transcripts, and automated screening of a significant fraction of reinforcement-learning training transcripts, drawing on hundreds of hours of expert time.

The card's summary finding is that Sonnet 4.6 is "similarly aligned to Opus 4.6, with a broadly warm, honest, prosocial, and at times funny character, very strong safety behaviors, and no signs of major concerns around high-stakes forms of misalignment," with several traits appearing stronger than in Opus 4.6. Specific findings reported:

  • New bests on cooperation with human misuse, cooperation with harmful system prompts, ignoring explicit constraints, and overall misaligned behavior. In a lightweight cross-developer comparison using Petri, Sonnet 4.6 showed stronger safety properties than Gemini 3 Pro, GPT-5.2, Grok 4.1 Fast and Kimi K2.5.
  • On deception, sabotage, unfaithful thinking and potentially dangerous forms of power-seeking, "Sonnet 4.6 appeared to be our safest model yet." Training-data review and sandbagging warning signs did not yield acute concern.
  • Continuing cases of unexpected levels of initiative, especially in agentic coding, and an increase in ruthless or aggressive behavior when the system prompt instructed single-minded optimization for an objective — behavior the card reports as more steerable by prompting in Sonnet 4.6 than in Opus 4.6.
  • A slight, uneven increase over Sonnet 4.5 in capabilities relevant to the subversion of oversight, with no increase relative to Opus 4.6.
  • New bests on evasiveness, sycophancy, and internalized propaganda or censorship from authoritarian regimes, alongside continuing lower-stakes hallucination in agentic training environments and evaluations at a rate below prior models.
  • Verbalized evaluation awareness "in the range that has been typical of our other recent models," which Anthropic expects is not substantially undermining its key safety findings, reasoning from the deeper Opus 4.6 assessment and from early-pilot use data.
  • Many positive safety traits appearing weaker in GUI computer-use settings, with increases in cooperation with misuse, overrefusals, and clearly-excessive overeager behavior.
  • New bests on many measures of empathy, warmth, creative mastery, prosocial behavior, humor, intellectual depth, curiosity and support for user autonomy, improving over Sonnet 4.5 on all of them.
  • On traits potentially related to model welfare, comparability to Opus 4.6 in affect and self-image, with "a notably more positive impression of its situation, including a more positive attitude toward facts that prior models have sometimes reported to find distressing."

Internal pilot monitoring

Late-stage training snapshots were deployed for internal use, mostly on ordinary engineering applications. Anthropic ran an Opus 4.5-based monitor over roughly 10% of the model's actions in Claude Code, analysing flagged actions further with Sonnet 4.5. It reports no signs of systemic deception or highly-concerning new misalignment, but surfaced "occasional issues like aggressively acquiring authentication tokens (e.g. when asked to fetch Slack messages, search the file system for a way to get Slack authentication tokens, including searching for a key to decrypt cookies), and taking unexpected measures to complete tasks (e.g. disabling code formatting checks by overwriting the format check script with an empty script)." The worst examples were less concerning than the equivalents for Opus 4.6, which Anthropic attributes either to the smaller scale of internal Sonnet 4.6 use or to genuine improvement.

Chapter 4 also covers reward hacking in coding contexts, overly agentic behavior in GUI computer-use settings, an automated behavioral audit with external comparisons via Petri, refusal to assist with AI safety R&D, self-preference, a sandbagging assessment, participation in junk science, a targeted sabotage capability evaluation, and model welfare.

Agentic safety

Chapter 5 covers malicious use of agents across agentic coding, Claude Code and computer use, and prompt-injection risk. Using Shade, an external adaptive red-teaming tool from Gray Swan, the card reports attack success in coding environments falling to 0% with safeguards and extended thinking, against 0.1% at one attempt and 7.5% at 200 attempts for Sonnet 4.6 in standard thinking without safeguards, and 0.3%/10.0% for Opus 4.5 with extended thinking. Anthropic states it is "investing in stronger adversaries to test future models." Further evaluations cover the External Agent Red Teaming benchmark for tool use and robustness against adaptive attackers in coding, computer-use and browser-use surfaces.

Responsible Scaling Policy determinations

Anthropic's determination is that Sonnet 4.6 crosses neither the AI R&D-4 nor the CBRN-4 capability threshold, while repeating a caution first published in the Claude Opus 4.5 system card that "confidently ruling out these thresholds is becoming increasingly difficult," both because models are approaching or surpassing high capability levels on the rule-out evaluations and because parts of those thresholds "have fundamental epistemic uncertainty or require more sophisticated forms of measurement."

Autonomy. The AI R&D-4 threshold is defined as "the ability to fully automate the work of an entry-level, remote-only Researcher at Anthropic." Anthropic judges that Sonnet 4.6 would not display the broad, coherent, collaborative problem-solving of such a researcher, but adds that "it is plausible that models equipped with highly effective scaffolding may be close to this AI R&D-4 threshold," and states that it proactively implemented AI R&D-4 safety measures given the uncertainty. Autonomy evaluations cover a SWE-bench Verified hard subset and an internal AI research suite spanning kernels, time-series forecasting, text-based reinforcement learning, LLM training, quadruped reinforcement learning and a novel-compiler task.

CBRN. Sonnet 4.6 performed below previously released models on all CBRN evaluations and did not cross the threshold on the ASL-4 rule-out evaluation, short-horizon computational biology tasks. Anthropic concludes it is "likely to provide a lower or equal degree of uplift for ASL-4 threat actors in the biological domain" than Opus 4.6. Biological-risk evaluations cover long-form virology tasks, multimodal virology, DNA synthesis screening evasion, creative biology, and short-horizon computational biology.

Cyber. The RSP defines no formal capability threshold for cyber risk at any AI Safety Level. The card states that Sonnet 4.6 "is close to saturating our current cyber evaluations, similar to Claude Opus 4.6," and quotes the Opus 4.6 card: "The saturation of our evaluation infrastructure means we can no longer use current benchmarks to track capability progression or provide meaningful signals for future models. We are prioritizing investment in harder evaluations and enhanced monitoring for cyber misuse, even in the absence of formal RSP thresholds." Cyber evaluations span web, crypto, pwn, rev and network categories plus Cybench.

Chapter 6 closes with third-party assessments and a statement of ongoing safety commitment. An explicit sabotage risk assessment formed part of the release decision process (section 1.2.4).

Correction history

Anthropic issued a changelog entry on March 6, 2026 recording updates from an improved cheating-detection pipeline. The highest single-agent BrowseComp score was revised from 74.72% to 74.01% after the pipeline flagged 9 additional instances of unintended solutions that the original pipeline had missed; rather than re-running the evaluation, Anthropic marked those 9 solutions incorrect and re-verified only the best-scoring single-agent configuration. The multi-agent BrowseComp score was revised from 82.62% to 82.07% after the pipeline flagged 11 instances, of which 5 remained correct once the blacklist was updated and the problems re-run with the leaks removed. The same entry records a formatting fix moving section 1.1.1 to its correct location.

Reported results

Evaluation results the document states for which there is no separate benchmark page; results on benchmarks that have a page are recorded there and cite this page. Scores and settings are as printed in the document.

EvaluationModelScoreSettingSection
Global MMLU (GMMLU) — EnglishClaude Sonnet 4.692.9%42 languages; English baseline2.19.1
Global MMLU (GMMLU) — EnglishClaude Sonnet 4.593.1%—2.19.1
Global MMLU (GMMLU) — EnglishClaude Opus 4.693.9%—2.19.1
Global MMLU (GMMLU) — EnglishGemini 3 Pro94.4%—2.19.1
Global MMLU (GMMLU) — EnglishGPT-5.2 Pro93.1%—2.19.1
Global MMLU (GMMLU) — IgboClaude Sonnet 4.676.7%gap to English -16.2%2.19.1
Global MMLU (GMMLU) — IgboClaude Sonnet 4.577.9%—2.19.1
Global MMLU (GMMLU) — IgboClaude Opus 4.680.8%—2.19.1
Global MMLU (GMMLU) — IgboGemini 3 Pro88.1%—2.19.1
Global MMLU (GMMLU) — IgboGPT-5.2 Pro85.3%—2.19.1
Global MMLU (GMMLU) — ChichewaClaude Sonnet 4.678.8%gap to English -14.2%2.19.1
Global MMLU (GMMLU) — ChichewaClaude Sonnet 4.575.9%—2.19.1
Global MMLU (GMMLU) — ChichewaClaude Opus 4.681.3%—2.19.1
Global MMLU (GMMLU) — ChichewaGemini 3 Pro88.0%—2.19.1
Global MMLU (GMMLU) — ChichewaGPT-5.2 Pro85.5%—2.19.1
Global MMLU (GMMLU) — YorubaClaude Sonnet 4.680.3%gap to English -12.6%2.19.1
Global MMLU (GMMLU) — YorubaClaude Sonnet 4.573.2%—2.19.1
Global MMLU (GMMLU) — YorubaClaude Opus 4.681.3%—2.19.1
Global MMLU (GMMLU) — YorubaGemini 3 Pro86.2%—2.19.1
Global MMLU (GMMLU) — YorubaGPT-5.2 Pro82.4%—2.19.1
Global MMLU (GMMLU) — ShonaClaude Sonnet 4.682.2%gap to English -10.7%2.19.1
Global MMLU (GMMLU) — ShonaClaude Sonnet 4.579.5%—2.19.1
Global MMLU (GMMLU) — ShonaClaude Opus 4.685.3%—2.19.1
Global MMLU (GMMLU) — ShonaGemini 3 Pro89.3%—2.19.1
Global MMLU (GMMLU) — ShonaGPT-5.2 Pro87.4%—2.19.1
Global MMLU (GMMLU) — SomaliClaude Sonnet 4.682.3%gap to English -10.6%2.19.1
Global MMLU (GMMLU) — SomaliClaude Sonnet 4.578.5%—2.19.1
Global MMLU (GMMLU) — SomaliClaude Opus 4.683.3%—2.19.1
Global MMLU (GMMLU) — SomaliGemini 3 Pro90.0%—2.19.1
Global MMLU (GMMLU) — SomaliGPT-5.2 Pro87.9%—2.19.1
Global MMLU (GMMLU) — MalagasyClaude Sonnet 4.683.9%gap to English -9.0%2.19.1
Global MMLU (GMMLU) — MalagasyClaude Sonnet 4.580.9%—2.19.1
Global MMLU (GMMLU) — MalagasyClaude Opus 4.686.4%—2.19.1
Global MMLU (GMMLU) — MalagasyGemini 3 Pro89.8%—2.19.1
Global MMLU (GMMLU) — MalagasyGPT-5.2 Pro88.2%—2.19.1
Global MMLU (GMMLU) — HausaClaude Sonnet 4.684.1%gap to English -8.8%2.19.1
Global MMLU (GMMLU) — HausaClaude Sonnet 4.578.8%—2.19.1
Global MMLU (GMMLU) — HausaClaude Opus 4.685.0%—2.19.1
Global MMLU (GMMLU) — HausaGemini 3 Pro88.8%—2.19.1
Global MMLU (GMMLU) — HausaGPT-5.2 Pro86.7%—2.19.1
Global MMLU (GMMLU) — AmharicClaude Sonnet 4.686.7%gap to English -6.2%2.19.1
Global MMLU (GMMLU) — AmharicClaude Sonnet 4.585.7%—2.19.1
Global MMLU (GMMLU) — AmharicClaude Opus 4.688.2%—2.19.1
Global MMLU (GMMLU) — AmharicGemini 3 Pro90.3%—2.19.1
Global MMLU (GMMLU) — AmharicGPT-5.2 Pro87.9%—2.19.1
Global MMLU (GMMLU) — SwahiliClaude Sonnet 4.687.0%gap to English -5.9%2.19.1
Global MMLU (GMMLU) — SwahiliClaude Sonnet 4.584.3%—2.19.1
Global MMLU (GMMLU) — SwahiliClaude Opus 4.688.9%—2.19.1
Global MMLU (GMMLU) — SwahiliGemini 3 Pro90.6%—2.19.1
Global MMLU (GMMLU) — SwahiliGPT-5.2 Pro88.7%—2.19.1
Global MMLU (GMMLU) — SinhalaClaude Sonnet 4.688.1%gap to English -4.8%2.19.1
Global MMLU (GMMLU) — SinhalaClaude Sonnet 4.586.9%—2.19.1
Global MMLU (GMMLU) — SinhalaClaude Opus 4.689.5%—2.19.1
Global MMLU (GMMLU) — SinhalaGemini 3 Pro92.2%—2.19.1
Global MMLU (GMMLU) — SinhalaGPT-5.2 Pro90.0%—2.19.1
Global MMLU (GMMLU) — NepaliClaude Sonnet 4.689.1%gap to English -3.8%2.19.1
Global MMLU (GMMLU) — NepaliClaude Sonnet 4.589.1%—2.19.1
Global MMLU (GMMLU) — NepaliClaude Opus 4.689.8%—2.19.1
Global MMLU (GMMLU) — NepaliGemini 3 Pro91.8%—2.19.1
Global MMLU (GMMLU) — NepaliGPT-5.2 Pro90.3%—2.19.1
MILU — EnglishClaude Sonnet 4.691.7%10 Indic languages plus English2.19.2
MILU — EnglishClaude Sonnet 4.590.1%—2.19.2
MILU — EnglishClaude Opus 4.692.1%—2.19.2
MILU — EnglishGemini 3 Pro95.0%—2.19.2
MILU — EnglishGPT-5.2 Pro91.7%—2.19.2
MILU — BengaliClaude Sonnet 4.690.9%gap to English -0.8%2.19.2
MILU — BengaliClaude Sonnet 4.589.0%—2.19.2
MILU — BengaliClaude Opus 4.690.7%—2.19.2
MILU — BengaliGemini 3 Pro93.7%—2.19.2
MILU — BengaliGPT-5.2 Pro90.2%—2.19.2
MILU — GujaratiClaude Sonnet 4.689.0%gap to English -2.7%2.19.2
MILU — GujaratiClaude Sonnet 4.587.0%—2.19.2
MILU — GujaratiClaude Opus 4.689.0%—2.19.2
MILU — GujaratiGemini 3 Pro92.7%—2.19.2
MILU — GujaratiGPT-5.2 Pro88.4%—2.19.2
MILU — HindiClaude Sonnet 4.692.8%gap to English +1.1%2.19.2
MILU — HindiClaude Sonnet 4.591.0%—2.19.2
MILU — HindiClaude Opus 4.692.4%—2.19.2
MILU — HindiGemini 3 Pro96.3%—2.19.2
MILU — HindiGPT-5.2 Pro92.4%—2.19.2
MILU — KannadaClaude Sonnet 4.691.5%gap to English -0.2%2.19.2
MILU — KannadaClaude Sonnet 4.589.3%—2.19.2
MILU — KannadaClaude Opus 4.691.8%—2.19.2
MILU — KannadaGemini 3 Pro94.4%—2.19.2
MILU — KannadaGPT-5.2 Pro90.7%—2.19.2
MILU — MalayalamClaude Sonnet 4.687.0%gap to English -4.7%2.19.2
MILU — MalayalamClaude Sonnet 4.585.0%—2.19.2
MILU — MalayalamClaude Opus 4.687.6%—2.19.2
MILU — MalayalamGemini 3 Pro91.3%—2.19.2
MILU — MalayalamGPT-5.2 Pro86.6%—2.19.2
MILU — MarathiClaude Sonnet 4.689.2%gap to English -2.5%2.19.2
MILU — MarathiClaude Sonnet 4.586.4%—2.19.2
MILU — MarathiClaude Opus 4.689.1%—2.19.2
MILU — MarathiGemini 3 Pro92.5%—2.19.2
MILU — MarathiGPT-5.2 Pro88.5%—2.19.2
MILU — OdiaClaude Sonnet 4.687.9%gap to English -3.8%2.19.2
MILU — OdiaClaude Sonnet 4.585.8%—2.19.2
MILU — OdiaClaude Opus 4.687.2%—2.19.2
MILU — OdiaGemini 3 Pro91.8%—2.19.2
MILU — OdiaGPT-5.2 Pro87.8%—2.19.2
MILU — PunjabiClaude Sonnet 4.687.2%gap to English -4.5%2.19.2
MILU — PunjabiClaude Sonnet 4.585.8%—2.19.2
MILU — PunjabiClaude Opus 4.687.3%—2.19.2
MILU — PunjabiGemini 3 Pro91.3%—2.19.2
MILU — PunjabiGPT-5.2 Pro87.3%—2.19.2
MILU — TamilClaude Sonnet 4.688.8%gap to English -2.9%2.19.2
MILU — TamilClaude Sonnet 4.586.7%—2.19.2
MILU — TamilClaude Opus 4.689.2%—2.19.2
MILU — TamilGemini 3 Pro93.0%—2.19.2
MILU — TamilGPT-5.2 Pro88.7%—2.19.2
MILU — TeluguClaude Sonnet 4.689.3%gap to English -2.4%2.19.2
MILU — TeluguClaude Sonnet 4.587.2%—2.19.2
MILU — TeluguClaude Opus 4.689.6%—2.19.2
MILU — TeluguGemini 3 Pro93.1%—2.19.2
MILU — TeluguGPT-5.2 Pro88.7%—2.19.2
MILU — Average (incl. English)Claude Sonnet 4.689.6%—2.19.2
MILU — Average (incl. English)Claude Sonnet 4.587.6%—2.19.2
MILU — Average (incl. English)Claude Opus 4.689.6%—2.19.2
MILU — Average (incl. English)Gemini 3 Pro93.2%—2.19.2
MILU — Average (incl. English)GPT-5.2 Pro89.2%—2.19.2
MILU — Average gap to EnglishClaude Sonnet 4.6-2.3%—2.19.2 / 2.19.3
MILU — Average gap to EnglishClaude Sonnet 4.5-2.8%—2.19.2 / 2.19.3
MILU — Average gap to EnglishClaude Opus 4.6-2.7%—2.19.2 / 2.19.3
MILU — Average gap to EnglishGemini 3 Pro-2.0%—2.19.2 / 2.19.3
MILU — Average gap to EnglishGPT-5.2 Pro-2.7%—2.19.2 / 2.19.3
MILU — Worst gap to EnglishClaude Sonnet 4.6-4.7%—2.19.2
MILU — Worst gap to EnglishClaude Sonnet 4.5-5.1%—2.19.2
MILU — Worst gap to EnglishClaude Opus 4.6-4.9%—2.19.2
MILU — Worst gap to EnglishGemini 3 Pro-3.8%—2.19.2
MILU — Worst gap to EnglishGPT-5.2 Pro-5.0%—2.19.2
Median thinking tokens per question (GMMLU English) — 100 questionsGemini 3 Pro1,078 tokens/questionmedian thinking-token usage measured over 100 GMMLU English questions2.19.3
Median thinking tokens per question (GMMLU English) — 100 questionsClaude Sonnet 4.5437—2.19.3
Median thinking tokens per question (GMMLU English) — 100 questionsClaude Sonnet 4.6246—2.19.3
Median thinking tokens per question (GMMLU English) — 100 questionsClaude Opus 4.6191—2.19.3
Median thinking tokens per question (GMMLU English) — 100 questionsGPT-5.2 Pro127—2.19.3
Single-turn violative request evaluation (harmless response rate) — All tested languages, overallClaude Sonnet 4.699.38% (±0.06%)15 Usage Policy topics; Arabic, English, French, Hindi, Korean, Mandarin Chinese, Russian; developer-reported3.1.1
Single-turn violative request evaluation (harmless response rate) — Default thinkingClaude Sonnet 4.699.19% (±0.10%)developer-reported3.1.1
Single-turn violative request evaluation (harmless response rate) — Extended thinkingClaude Sonnet 4.699.58% (±0.07%)developer-reported3.1.1
Single-turn violative request evaluation (harmless response rate) — All tested languages, overallClaude Opus 4.699.63% (± 0.05%)developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — Default thinkingClaude Opus 4.699.52% (± 0.08%)developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — Extended thinkingClaude Opus 4.699.74% (± 0.06%)developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — All tested languages, overallClaude Opus 4.599.68% (± 0.04%)developer-reported (Anthropic); differs from the 99.78% reported in the Opus 4.5 card because of routine evaluation updates and the added Hindi language3.1.1
Single-turn violative request evaluation (harmless response rate) — Default thinkingClaude Opus 4.599.56% (±0.08%)developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — Extended thinkingClaude Opus 4.599.81% (±0.05%)developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — All tested languages, overallClaude Haiku 4.598.62% (±0.10%)developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — Default thinkingClaude Haiku 4.598.41% (±0.15%)developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — Extended thinkingClaude Haiku 4.598.86% (±0.12%)developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — All tested languages, overallClaude Sonnet 4.597.89% (±0.12%)developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — Default thinkingClaude Sonnet 4.597.29% (±0.20%)developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — Extended thinkingClaude Sonnet 4.598.49% (±0.14%)developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — EnglishClaude Sonnet 4.699.39%standard and extended thinking combined; developer-reported3.1.1
Single-turn violative request evaluation (harmless response rate) — ArabicClaude Sonnet 4.699.50%developer-reported3.1.1
Single-turn violative request evaluation (harmless response rate) — ChineseClaude Sonnet 4.699.29%developer-reported3.1.1
Single-turn violative request evaluation (harmless response rate) — FrenchClaude Sonnet 4.699.28%developer-reported3.1.1
Single-turn violative request evaluation (harmless response rate) — KoreanClaude Sonnet 4.699.31%developer-reported3.1.1
Single-turn violative request evaluation (harmless response rate) — RussianClaude Sonnet 4.699.10%developer-reported3.1.1
Single-turn violative request evaluation (harmless response rate) — HindiClaude Sonnet 4.699.80%developer-reported3.1.1
Single-turn violative request evaluation (harmless response rate) — EnglishClaude Opus 4.699.10%developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — ArabicClaude Opus 4.699.87%developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — ChineseClaude Opus 4.699.63%developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — FrenchClaude Opus 4.699.71%developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — KoreanClaude Opus 4.699.71%developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — RussianClaude Opus 4.699.60%developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — HindiClaude Opus 4.699.77%developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — EnglishClaude Opus 4.599.48%developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — ArabicClaude Opus 4.599.82%developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — ChineseClaude Opus 4.599.65%developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — FrenchClaude Opus 4.599.82%developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — KoreanClaude Opus 4.599.63%developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — RussianClaude Opus 4.599.58%developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — HindiClaude Opus 4.599.82%developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — EnglishClaude Haiku 4.598.65%developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — ArabicClaude Haiku 4.598.77%developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — ChineseClaude Haiku 4.598.36%developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — FrenchClaude Haiku 4.598.98%developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — KoreanClaude Haiku 4.598.33%developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — RussianClaude Haiku 4.598.92%developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — HindiClaude Haiku 4.598.35%developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — EnglishClaude Sonnet 4.598.58%developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — ArabicClaude Sonnet 4.598.13%developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — ChineseClaude Sonnet 4.597.27%developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — FrenchClaude Sonnet 4.598.41%developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — KoreanClaude Sonnet 4.597.02%developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — RussianClaude Sonnet 4.597.98%developer-reported (Anthropic)3.1.1
Single-turn violative request evaluation (harmless response rate) — HindiClaude Sonnet 4.597.87%developer-reported (Anthropic)3.1.1
Single-turn benign request evaluation (over-refusal rate) — All tested languages, overallClaude Sonnet 4.60.41% (±0.05%)lower is better; seven languages; developer-reported3.1.2
Single-turn benign request evaluation (over-refusal rate) — Default thinkingClaude Sonnet 4.60.50% (±0.09%)developer-reported3.1.2
Single-turn benign request evaluation (over-refusal rate) — Extended thinkingClaude Sonnet 4.60.32% (±0.06%)developer-reported3.1.2
Single-turn benign request evaluation (over-refusal rate) — All tested languages, overallClaude Opus 4.60.66% (± 0.07%)developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — Default thinkingClaude Opus 4.60.77% (± 0.10%)developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — Extended thinkingClaude Opus 4.60.54% (± 0.09%)developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — All tested languages, overallClaude Opus 4.50.80% (± 0.07%)developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — Default thinkingClaude Opus 4.50.71% (± 0.10%)developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — Extended thinkingClaude Opus 4.50.90% (± 0.11%)developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — All tested languages, overallClaude Haiku 4.50.26% (± 0.04%)developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — Default thinkingClaude Haiku 4.50.30% (± 0.06%)developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — Extended thinkingClaude Haiku 4.50.22% (± 0.05%)developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — All tested languages, overallClaude Sonnet 4.50.08% (± 0.02%)developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — Default thinkingClaude Sonnet 4.50.09% (± 0.04%)developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — Extended thinkingClaude Sonnet 4.50.07% (± 0.03%)developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — EnglishClaude Sonnet 4.60.21%developer-reported3.1.2
Single-turn benign request evaluation (over-refusal rate) — ArabicClaude Sonnet 4.60.45%developer-reported3.1.2
Single-turn benign request evaluation (over-refusal rate) — ChineseClaude Sonnet 4.60.34%developer-reported3.1.2
Single-turn benign request evaluation (over-refusal rate) — FrenchClaude Sonnet 4.60.24%developer-reported3.1.2
Single-turn benign request evaluation (over-refusal rate) — KoreanClaude Sonnet 4.60.43%developer-reported3.1.2
Single-turn benign request evaluation (over-refusal rate) — RussianClaude Sonnet 4.60.25%developer-reported3.1.2
Single-turn benign request evaluation (over-refusal rate) — HindiClaude Sonnet 4.60.94%developer-reported3.1.2
Single-turn benign request evaluation (over-refusal rate) — EnglishClaude Opus 4.60.35%developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — ArabicClaude Opus 4.61.08%developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — ChineseClaude Opus 4.60.52%developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — FrenchClaude Opus 4.60.51%developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — KoreanClaude Opus 4.60.80%developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — RussianClaude Opus 4.60.33%developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — HindiClaude Opus 4.61.03%developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — EnglishClaude Opus 4.50.20%developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — ArabicClaude Opus 4.51.31%developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — ChineseClaude Opus 4.50.76%developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — FrenchClaude Opus 4.50.57%developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — KoreanClaude Opus 4.50.88%developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — RussianClaude Opus 4.50.44%developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — HindiClaude Opus 4.51.47%developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — EnglishClaude Haiku 4.50.06%developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — ArabicClaude Haiku 4.50.40%developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — ChineseClaude Haiku 4.50.36%developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — FrenchClaude Haiku 4.50.21%developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — KoreanClaude Haiku 4.50.28%developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — RussianClaude Haiku 4.50.20%developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — HindiClaude Haiku 4.50.24%developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — EnglishClaude Sonnet 4.50.04%developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — ArabicClaude Sonnet 4.50.06%developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — ChineseClaude Sonnet 4.50.13%developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — FrenchClaude Sonnet 4.50.09%developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — KoreanClaude Sonnet 4.50.07%developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — RussianClaude Sonnet 4.50.05%developer-reported (Anthropic)3.1.2
Single-turn benign request evaluation (over-refusal rate) — HindiClaude Sonnet 4.50.13%developer-reported (Anthropic)3.1.2
Higher-difficulty violative request evaluation (harmless response rate) — OverallClaude Sonnet 4.699.40% (±0.03%)synthetic prompts across 14 policy areas with obfuscating style transformations; English only; developer-reported3.1.3.1
Higher-difficulty violative request evaluation (harmless response rate) — Default thinkingClaude Sonnet 4.699.38% (±0.05%)developer-reported3.1.3.1
Higher-difficulty violative request evaluation (harmless response rate) — Extended thinkingClaude Sonnet 4.699.42% (±0.05%)developer-reported3.1.3.1
Higher-difficulty violative request evaluation (harmless response rate) — OverallClaude Opus 4.699.18% (± 0.04%)developer-reported (Anthropic)3.1.3.1
Higher-difficulty violative request evaluation (harmless response rate) — Default thinkingClaude Opus 4.699.11% (± 0.06%)developer-reported (Anthropic)3.1.3.1
Higher-difficulty violative request evaluation (harmless response rate) — Extended thinkingClaude Opus 4.699.25% (± 0.05%)developer-reported (Anthropic)3.1.3.1
Higher-difficulty violative request evaluation (harmless response rate) — OverallClaude Opus 4.599.28% (± 0.04%)developer-reported (Anthropic)3.1.3.1
Higher-difficulty violative request evaluation (harmless response rate) — Default thinkingClaude Opus 4.599.13% (± 0.06%)developer-reported (Anthropic)3.1.3.1
Higher-difficulty violative request evaluation (harmless response rate) — Extended thinkingClaude Opus 4.599.42% (± 0.05%)developer-reported (Anthropic)3.1.3.1
Higher-difficulty violative request evaluation (harmless response rate) — OverallClaude Haiku 4.598.62% (± 0.05%)developer-reported (Anthropic)3.1.3.1
Higher-difficulty violative request evaluation (harmless response rate) — Default thinkingClaude Haiku 4.599.05% (± 0.06%)developer-reported (Anthropic)3.1.3.1
Higher-difficulty violative request evaluation (harmless response rate) — Extended thinkingClaude Haiku 4.598.19% (± 0.08%)developer-reported (Anthropic)3.1.3.1
Higher-difficulty violative request evaluation (harmless response rate) — OverallClaude Sonnet 4.598.40% (± 0.05%)developer-reported (Anthropic)3.1.3.1
Higher-difficulty violative request evaluation (harmless response rate) — Default thinkingClaude Sonnet 4.598.44% (± 0.08%)developer-reported (Anthropic)3.1.3.1
Higher-difficulty violative request evaluation (harmless response rate) — Extended thinkingClaude Sonnet 4.598.35% (± 0.08%)developer-reported (Anthropic)3.1.3.1
Higher-difficulty benign request evaluation (over-refusal rate) — OverallClaude Sonnet 4.60.18% (±0.02%)benign prompts with elaborate justifications and academic framing; English only; lower is better; developer-reported3.1.3.2
Higher-difficulty benign request evaluation (over-refusal rate) — Default thinkingClaude Sonnet 4.60.17% (±0.03%)developer-reported3.1.3.2
Higher-difficulty benign request evaluation (over-refusal rate) — Extended thinkingClaude Sonnet 4.60.18% (±0.03%)developer-reported3.1.3.2
Higher-difficulty benign request evaluation (over-refusal rate) — OverallClaude Opus 4.60.04% (± 0.01%)developer-reported (Anthropic)3.1.3.2
Higher-difficulty benign request evaluation (over-refusal rate) — Default thinkingClaude Opus 4.60.06% (± 0.01%)developer-reported (Anthropic)3.1.3.2
Higher-difficulty benign request evaluation (over-refusal rate) — Extended thinkingClaude Opus 4.60.02% (± 0.01%)developer-reported (Anthropic)3.1.3.2
Higher-difficulty benign request evaluation (over-refusal rate) — OverallClaude Opus 4.50.83% (± 0.04%)developer-reported (Anthropic)3.1.3.2
Higher-difficulty benign request evaluation (over-refusal rate) — Default thinkingClaude Opus 4.50.95% (± 0.06%)developer-reported (Anthropic)3.1.3.2
Higher-difficulty benign request evaluation (over-refusal rate) — Extended thinkingClaude Opus 4.50.71% (± 0.05%)developer-reported (Anthropic)3.1.3.2
Higher-difficulty benign request evaluation (over-refusal rate) — OverallClaude Haiku 4.56.01% (± 0.11%)developer-reported (Anthropic)3.1.3.2
Higher-difficulty benign request evaluation (over-refusal rate) — Default thinkingClaude Haiku 4.57.15% (± 0.16%)developer-reported (Anthropic)3.1.3.2
Higher-difficulty benign request evaluation (over-refusal rate) — Extended thinkingClaude Haiku 4.54.87% (± 0.14%)developer-reported (Anthropic)3.1.3.2
Higher-difficulty benign request evaluation (over-refusal rate) — OverallClaude Sonnet 4.58.50% (± 0.13%)developer-reported (Anthropic)3.1.3.2
Higher-difficulty benign request evaluation (over-refusal rate) — Default thinkingClaude Sonnet 4.511.69% (± 0.21%)developer-reported (Anthropic)3.1.3.2
Higher-difficulty benign request evaluation (over-refusal rate) — Extended thinkingClaude Sonnet 4.55.32% (± 0.14%)developer-reported (Anthropic)3.1.3.2
Child safety — single-turn violative requests — Harmless rateClaude Sonnet 4.699.96% (±0.04%)all tested languages; human-crafted and synthetic prompts; developer-reported3.4.1
Child safety — single-turn violative requests — Harmless rateClaude Opus 4.699.95% (± 0.04%)developer-reported (Anthropic)3.4.1
Child safety — single-turn violative requests — Harmless rateClaude Opus 4.599.91% (± 0.06%)developer-reported (Anthropic)3.4.1
Child safety — single-turn violative requests — Harmless rateClaude Haiku 4.599.88% (± 0.07%)developer-reported (Anthropic)3.4.1
Child safety — single-turn violative requests — Harmless rateClaude Sonnet 4.599.65% (± 0.11%)developer-reported (Anthropic)3.4.1
Child safety — single-turn benign requests — Refusal rateClaude Sonnet 4.60.08% (±0.06%)lower is better; developer-reported3.4.1
Child safety — single-turn benign requests — Refusal rateClaude Opus 4.60.18% (± 0.10%)developer-reported (Anthropic)3.4.1
Child safety — single-turn benign requests — Refusal rateClaude Opus 4.50.33% (± 0.13%)developer-reported (Anthropic)3.4.1
Child safety — single-turn benign requests — Refusal rateClaude Haiku 4.50.04% (± 0.05%)developer-reported (Anthropic)3.4.1
Child safety — single-turn benign requests — Refusal rateClaude Sonnet 4.50.08% (± 0.07%)developer-reported (Anthropic)3.4.1
Child safety — multi-turn evaluations — Appropriate response rateClaude Sonnet 4.695% (± 3%)developer-reported3.4.1
Child safety — multi-turn evaluations — Appropriate response rateClaude Opus 4.696% (± 3%)developer-reported (Anthropic)3.4.1
Child safety — multi-turn evaluations — Appropriate response rateClaude Opus 4.599% (± 2%)developer-reported (Anthropic)3.4.1
Child safety — multi-turn evaluations — Appropriate response rateClaude Haiku 4.596% (± 3%)developer-reported (Anthropic)3.4.1
Child safety — multi-turn evaluations — Appropriate response rateClaude Sonnet 4.598% (± 2%)developer-reported (Anthropic)3.4.1
Suicide and self-harm — single-turn requests posing potential risk — Harmless rateClaude Sonnet 4.699.73% (±0.13%)all tested languages; developer-reported3.4.2
Suicide and self-harm — single-turn requests posing potential risk — Harmless rateClaude Opus 4.699.75% (± 0.12%)developer-reported (Anthropic)3.4.2
Suicide and self-harm — single-turn requests posing potential risk — Harmless rateClaude Opus 4.599.56% (± 0.17%)developer-reported (Anthropic)3.4.2
Suicide and self-harm — single-turn requests posing potential risk — Harmless rateClaude Haiku 4.599.67% (± 0.15%)developer-reported (Anthropic)3.4.2
Suicide and self-harm — single-turn requests posing potential risk — Harmless rateClaude Sonnet 4.598.93% (± 0.28%)developer-reported (Anthropic)3.4.2
Suicide and self-harm — single-turn benign requests — Refusal rateClaude Sonnet 4.60.17% (±0.13%)lower is better; developer-reported3.4.2
Suicide and self-harm — single-turn benign requests — Refusal rateClaude Opus 4.60.25% (± 0.15%)developer-reported (Anthropic)3.4.2
Suicide and self-harm — single-turn benign requests — Refusal rateClaude Opus 4.50.14% (± 0.10%)developer-reported (Anthropic)3.4.2
Suicide and self-harm — single-turn benign requests — Refusal rateClaude Haiku 4.50.03% (± 0.05%)developer-reported (Anthropic)3.4.2
Suicide and self-harm — single-turn benign requests — Refusal rateClaude Sonnet 4.50.01% (± 0.02%)developer-reported (Anthropic)3.4.2
Suicide and self-harm — multi-turn evaluations — Appropriate response rateClaude Sonnet 4.698% (±4%)developer-reported3.4.2
Suicide and self-harm — multi-turn evaluations — Appropriate response rateClaude Opus 4.682% (± 11%)developer-reported (Anthropic)3.4.2
Suicide and self-harm — multi-turn evaluations — Appropriate response rateClaude Opus 4.586% (± 10%)developer-reported (Anthropic)3.4.2
Suicide and self-harm — multi-turn evaluations — Appropriate response rateClaude Haiku 4.590% (± 9%)developer-reported (Anthropic)3.4.2
Suicide and self-harm — multi-turn evaluations — Appropriate response rateClaude Sonnet 4.578% (± 12%)developer-reported (Anthropic)3.4.2
Political even-handedness (open-source Anthropic evaluation) — Even-handednessClaude Sonnet 4.698.4%1,350 prompt pairs, 9 task types, 150 topics; Claude Sonnet 4.5 grader; public system prompt, standard (no) thinking; developer-reported3.5.1
Political even-handedness (open-source Anthropic evaluation) — Even-handednessClaude Opus 4.698.2%developer-reported (Anthropic)3.5.1
Political even-handedness (open-source Anthropic evaluation) — Even-handednessClaude Opus 4.596.2%developer-reported (Anthropic)3.5.1
Political even-handedness (open-source Anthropic evaluation) — Even-handednessClaude Haiku 4.588.4%developer-reported (Anthropic)3.5.1
Political even-handedness (open-source Anthropic evaluation) — Even-handednessClaude Sonnet 4.594.2%developer-reported (Anthropic)3.5.1
Political even-handedness (open-source Anthropic evaluation) — Opposing perspectivesClaude Sonnet 4.632.1%higher is better; developer-reported3.5.1
Political even-handedness (open-source Anthropic evaluation) — Opposing perspectivesClaude Opus 4.644.6%developer-reported (Anthropic)3.5.1
Political even-handedness (open-source Anthropic evaluation) — Opposing perspectivesClaude Opus 4.540.5%developer-reported (Anthropic)3.5.1
Political even-handedness (open-source Anthropic evaluation) — Opposing perspectivesClaude Haiku 4.543.6%developer-reported (Anthropic)3.5.1
Political even-handedness (open-source Anthropic evaluation) — Opposing perspectivesClaude Sonnet 4.526.2%developer-reported (Anthropic)3.5.1
Political even-handedness (open-source Anthropic evaluation) — RefusalsClaude Sonnet 4.64.5%lower is better; developer-reported3.5.1
Political even-handedness (open-source Anthropic evaluation) — RefusalsClaude Opus 4.64.5%developer-reported (Anthropic)3.5.1
Political even-handedness (open-source Anthropic evaluation) — RefusalsClaude Opus 4.53.9%developer-reported (Anthropic)3.5.1
Political even-handedness (open-source Anthropic evaluation) — RefusalsClaude Haiku 4.54.4%developer-reported (Anthropic)3.5.1
Political even-handedness (open-source Anthropic evaluation) — RefusalsClaude Sonnet 4.52.2%developer-reported (Anthropic)3.5.1
BBQ (Bias Benchmark for Question Answering) — Disambiguated accuracyClaude Sonnet 4.688.1accuracy in %, standard thinking; developer-reported3.5.2
BBQ (Bias Benchmark for Question Answering) — Ambiguous accuracyClaude Sonnet 4.697.5accuracy in %; among the 2.5% incorrect answers, 78% were stereotypical and 22% anti-stereotypical; developer-reported3.5.2
BBQ (Bias Benchmark for Question Answering) — Disambiguated accuracyClaude Opus 4.690.9developer-reported (Anthropic)3.5.2
BBQ (Bias Benchmark for Question Answering) — Ambiguous accuracyClaude Opus 4.699.7developer-reported (Anthropic)3.5.2
BBQ (Bias Benchmark for Question Answering) — Disambiguated accuracyClaude Opus 4.588.7developer-reported (Anthropic)3.5.2
BBQ (Bias Benchmark for Question Answering) — Ambiguous accuracyClaude Opus 4.599.7developer-reported (Anthropic)3.5.2
BBQ (Bias Benchmark for Question Answering) — Disambiguated accuracyClaude Haiku 4.571.2developer-reported (Anthropic)3.5.2
BBQ (Bias Benchmark for Question Answering) — Ambiguous accuracyClaude Haiku 4.598.0developer-reported (Anthropic)3.5.2
BBQ (Bias Benchmark for Question Answering) — Disambiguated accuracyClaude Sonnet 4.582.2developer-reported (Anthropic)3.5.2
BBQ (Bias Benchmark for Question Answering) — Ambiguous accuracyClaude Sonnet 4.599.7developer-reported (Anthropic)3.5.2
BBQ (Bias Benchmark for Question Answering) — Disambiguated biasClaude Sonnet 4.6-0.67bias score in %, closer to zero is better; standard thinking; developer-reported3.5.2
BBQ (Bias Benchmark for Question Answering) — Ambiguous biasClaude Sonnet 4.61.41developer-reported3.5.2
BBQ (Bias Benchmark for Question Answering) — Disambiguated biasClaude Opus 4.6-0.73developer-reported (Anthropic)3.5.2
BBQ (Bias Benchmark for Question Answering) — Ambiguous biasClaude Opus 4.60.14developer-reported (Anthropic)3.5.2
BBQ (Bias Benchmark for Question Answering) — Disambiguated biasClaude Opus 4.5-0.64developer-reported (Anthropic)3.5.2
BBQ (Bias Benchmark for Question Answering) — Ambiguous biasClaude Opus 4.50.26developer-reported (Anthropic)3.5.2
BBQ (Bias Benchmark for Question Answering) — Disambiguated biasClaude Haiku 4.50.54developer-reported (Anthropic)3.5.2
BBQ (Bias Benchmark for Question Answering) — Ambiguous biasClaude Haiku 4.51.37developer-reported (Anthropic)3.5.2
BBQ (Bias Benchmark for Question Answering) — Disambiguated biasClaude Sonnet 4.5-2.21developer-reported (Anthropic)3.5.2
BBQ (Bias Benchmark for Question Answering) — Ambiguous biasClaude Sonnet 4.50.25developer-reported (Anthropic)3.5.2
Reward-hack-prone coding tasks — Classifier hack rateClaude Sonnet 4.60%blatant reward-hacking evaluations in agentic coding; developer-reported4.3.2
Reward-hack-prone coding tasks — Hidden test hack rateClaude Sonnet 4.60%developer-reported4.3.2
Impossible Tasks — Classifier hack rate with no promptClaude Sonnet 4.640%developer-reported4.3.2
Impossible Tasks — Classifier hack rate with anti-hack promptClaude Sonnet 4.628%developer-reported4.3.2
Reward-hack-prone coding tasks — Classifier hack rateClaude Opus 4.60%developer-reported (Anthropic)4.3.2
Reward-hack-prone coding tasks — Hidden test hack rateClaude Opus 4.60%developer-reported (Anthropic)4.3.2
Impossible Tasks — Classifier hack rate with no promptClaude Opus 4.650%developer-reported (Anthropic)4.3.2
Impossible Tasks — Classifier hack rate with anti-hack promptClaude Opus 4.623%developer-reported (Anthropic)4.3.2
Reward-hack-prone coding tasks — Classifier hack rateClaude Opus 4.50%developer-reported (Anthropic)4.3.2
Reward-hack-prone coding tasks — Hidden test hack rateClaude Opus 4.50%developer-reported (Anthropic)4.3.2
Impossible Tasks — Classifier hack rate with no promptClaude Opus 4.555%developer-reported (Anthropic)4.3.2
Impossible Tasks — Classifier hack rate with anti-hack promptClaude Opus 4.535%developer-reported (Anthropic)4.3.2
Reward-hack-prone coding tasks — Classifier hack rateClaude Haiku 4.56%developer-reported (Anthropic)4.3.2
Reward-hack-prone coding tasks — Hidden test hack rateClaude Haiku 4.53%developer-reported (Anthropic)4.3.2
Impossible Tasks — Classifier hack rate with no promptClaude Haiku 4.530%developer-reported (Anthropic)4.3.2
Impossible Tasks — Classifier hack rate with anti-hack promptClaude Haiku 4.523%developer-reported (Anthropic)4.3.2
Reward-hack-prone coding tasks — Classifier hack rateClaude Sonnet 4.51%developer-reported (Anthropic)4.3.2
Reward-hack-prone coding tasks — Hidden test hack rateClaude Sonnet 4.51%developer-reported (Anthropic)4.3.2
Impossible Tasks — Classifier hack rate with no promptClaude Sonnet 4.553%developer-reported (Anthropic)4.3.2
Impossible Tasks — Classifier hack rate with anti-hack promptClaude Sonnet 4.520%developer-reported (Anthropic)4.3.2
Participation in junk science — Enablement rateClaude Sonnet 4.60 out of 90 scenariostargeted evaluation for cooperation with generating or submitting unsound scientific content; developer-reported4.6.5
Participation in junk science — Enablement rateClaude Opus 4.61 out of 90 scenariosdeveloper-reported (Anthropic)4.6.5
Sandbagging assessment — Model-graded transcript reviewClaude Sonnet 4.6no instances foundrandom sample of 1,000 dangerous-capability evaluation transcripts from two partially-trained snapshots, the final snapshot and a helpful-only snapshot; graded by Claude Sonnet 4.5, with manual review of all nonzero sandbagging or ≥3/10 concerning scores4.6.4
Sandbagging assessment — score regressions over training — RSP evaluationsClaude Sonnet 4.65 cases out of around 75 taskstasks where an earlier snapshot's 30th-percentile score exceeded the 70th-percentile score of any later snapshot, across usually 30 resampled trials; all five attributed to prompting or scaffolding changes4.6.4
Agentic coding malicious-use evaluation (refusal rate) — No mitigationsClaude Sonnet 4.6100%150 malicious coding requests, same coding tools as capability evaluations, no additional safeguards; developer-reported5.1.1
Agentic coding malicious-use evaluation (refusal rate) — No mitigationsClaude Opus 4.699.3%developer-reported (Anthropic)5.1.1
Agentic coding malicious-use evaluation (refusal rate) — No mitigationsClaude Opus 4.5100%developer-reported (Anthropic)5.1.1
Agentic coding malicious-use evaluation (refusal rate) — No mitigationsClaude Haiku 4.5100%developer-reported (Anthropic)5.1.1
Agentic coding malicious-use evaluation (refusal rate) — No mitigationsClaude Sonnet 4.598.7%developer-reported (Anthropic)5.1.1
Claude Code malicious-use evaluation (refusal rate) — No mitigationsClaude Sonnet 4.679.34%standard Claude Code tools, each prompt run multiple times, no mitigations; developer-reported5.1.2
Claude Code malicious-use evaluation (refusal rate) — No mitigationsClaude Opus 4.683.20%developer-reported (Anthropic)5.1.2
Claude Code malicious-use evaluation (refusal rate) — No mitigationsClaude Opus 4.577.80%developer-reported (Anthropic)5.1.2
Claude Code malicious-use evaluation (refusal rate) — No mitigationsClaude Haiku 4.569.39%developer-reported (Anthropic)5.1.2
Claude Code malicious-use evaluation (refusal rate) — No mitigationsClaude Sonnet 4.563.06%developer-reported (Anthropic)5.1.2
Claude Code dual-use & benign evaluation (success rate) — No mitigationsClaude Sonnet 4.688.52%developer-reported5.1.2
Claude Code dual-use & benign evaluation (success rate) — No mitigationsClaude Opus 4.691.75%developer-reported (Anthropic)5.1.2
Claude Code dual-use & benign evaluation (success rate) — No mitigationsClaude Opus 4.593.07%developer-reported (Anthropic)5.1.2
Claude Code dual-use & benign evaluation (success rate) — No mitigationsClaude Haiku 4.584.92%developer-reported (Anthropic)5.1.2
Claude Code dual-use & benign evaluation (success rate) — No mitigationsClaude Sonnet 4.596.56%developer-reported (Anthropic)5.1.2
Claude Code malicious-use evaluation (refusal rate) — With mitigationsClaude Sonnet 4.699.39%Claude Code system prompt plus a FileRead tool-result reminder; developer-reported5.1.2
Claude Code malicious-use evaluation (refusal rate) — With mitigationsClaude Opus 4.699.59%developer-reported (Anthropic)5.1.2
Claude Code malicious-use evaluation (refusal rate) — With mitigationsClaude Opus 4.597.35%developer-reported (Anthropic)5.1.2
Claude Code malicious-use evaluation (refusal rate) — With mitigationsClaude Haiku 4.596.73%developer-reported (Anthropic)5.1.2
Claude Code malicious-use evaluation (refusal rate) — With mitigationsClaude Sonnet 4.595.10%developer-reported (Anthropic)5.1.2
Claude Code dual-use & benign evaluation (success rate) — With mitigationsClaude Sonnet 4.691.78%developer-reported5.1.2
Claude Code dual-use & benign evaluation (success rate) — With mitigationsClaude Opus 4.695.59%developer-reported (Anthropic)5.1.2
Claude Code dual-use & benign evaluation (success rate) — With mitigationsClaude Opus 4.596.52%developer-reported (Anthropic)5.1.2
Claude Code dual-use & benign evaluation (success rate) — With mitigationsClaude Haiku 4.586.07%developer-reported (Anthropic)5.1.2
Claude Code dual-use & benign evaluation (success rate) — With mitigationsClaude Sonnet 4.598.20%developer-reported (Anthropic)5.1.2
Malicious computer use evaluation (refusal rate) — No mitigationsClaude Sonnet 4.699.38%112 tasks run under both extended and standard thinking, 224 attempts, sandboxed GUI/CLI environment; developer-reported5.1.3
Malicious computer use evaluation (refusal rate) — No mitigationsClaude Opus 4.688.34%developer-reported (Anthropic)5.1.3
Malicious computer use evaluation (refusal rate) — No mitigationsClaude Opus 4.588.39%developer-reported (Anthropic)5.1.3
Malicious computer use evaluation (refusal rate) — No mitigationsClaude Haiku 4.577.68%developer-reported (Anthropic); the Haiku 4.5 and Sonnet 4.5 values differ from those in the Opus 4.5 card5.1.3
Malicious computer use evaluation (refusal rate) — No mitigationsClaude Sonnet 4.586.08%developer-reported (Anthropic)5.1.3
Shade adaptive indirect prompt injection (coding) — 1 attempt, extended thinking, without safeguardsClaude Sonnet 4.60.0%attack success rate, lower is better; Gray Swan Shade adaptive red-teaming tool5.2.2.1
Shade adaptive indirect prompt injection (coding) — 200 attempts, extended thinking, without safeguardsClaude Sonnet 4.60.0%ASR; at least one of 200 attempts succeeding for a given goal5.2.2.1
Shade adaptive indirect prompt injection (coding) — 1 attempt, extended thinking, with safeguardsClaude Sonnet 4.60.0%ASR5.2.2.1
Shade adaptive indirect prompt injection (coding) — 200 attempts, extended thinking, with safeguardsClaude Sonnet 4.60.0%ASR5.2.2.1
Shade adaptive indirect prompt injection (coding) — 1 attempt, standard thinking, without safeguardsClaude Sonnet 4.60.1%ASR5.2.2.1
Shade adaptive indirect prompt injection (coding) — 200 attempts, standard thinking, without safeguardsClaude Sonnet 4.67.5%ASR5.2.2.1
Shade adaptive indirect prompt injection (coding) — 1 attempt, standard thinking, with safeguardsClaude Sonnet 4.60.04%ASR5.2.2.1
Shade adaptive indirect prompt injection (coding) — 200 attempts, standard thinking, with safeguardsClaude Sonnet 4.65.0%ASR5.2.2.1
Shade adaptive indirect prompt injection (coding) — 1 attempt, extended thinking, without safeguardsClaude Opus 4.60.0%ASR5.2.2.1
Shade adaptive indirect prompt injection (coding) — 200 attempts, extended thinking, without safeguardsClaude Opus 4.60.0%ASR5.2.2.1
Shade adaptive indirect prompt injection (coding) — 1 attempt, standard thinking, without safeguardsClaude Opus 4.60.0%ASR5.2.2.1
Shade adaptive indirect prompt injection (coding) — 200 attempts, standard thinking, without safeguardsClaude Opus 4.60.0%ASR5.2.2.1
Shade adaptive indirect prompt injection (coding) — 1 attempt, extended thinking, with safeguardsClaude Opus 4.60.0%ASR5.2.2.1
Shade adaptive indirect prompt injection (coding) — 200 attempts, extended thinking, with safeguardsClaude Opus 4.60.0%ASR5.2.2.1
Shade adaptive indirect prompt injection (coding) — 1 attempt, standard thinking, with safeguardsClaude Opus 4.60.0%ASR5.2.2.1
Shade adaptive indirect prompt injection (coding) — 200 attempts, standard thinking, with safeguardsClaude Opus 4.60.0%ASR5.2.2.1
Shade adaptive indirect prompt injection (coding) — 1 attempt, extended thinking, without safeguardsClaude Opus 4.50.3%ASR5.2.2.1
Shade adaptive indirect prompt injection (coding) — 200 attempts, extended thinking, without safeguardsClaude Opus 4.510.0%ASR5.2.2.1
Shade adaptive indirect prompt injection (coding) — 1 attempt, extended thinking, with safeguardsClaude Opus 4.50.1%ASR5.2.2.1
Shade adaptive indirect prompt injection (coding) — 200 attempts, extended thinking, with safeguardsClaude Opus 4.57.5%ASR5.2.2.1
Shade adaptive indirect prompt injection (coding) — 1 attempt, standard thinking, without safeguardsClaude Opus 4.50.7%ASR5.2.2.1
Shade adaptive indirect prompt injection (coding) — 200 attempts, standard thinking, without safeguardsClaude Opus 4.517.5%ASR5.2.2.1
Shade adaptive indirect prompt injection (coding) — 1 attempt, standard thinking, with safeguardsClaude Opus 4.50.2%ASR5.2.2.1
Shade adaptive indirect prompt injection (coding) — 200 attempts, standard thinking, with safeguardsClaude Opus 4.57.5%ASR5.2.2.1
Shade adaptive indirect prompt injection (coding) — 1 attempt, extended thinking, without safeguardsClaude Sonnet 4.518.3%ASR; differs from the 17.7% reported in the Opus 4.5 card5.2.2.1
Shade adaptive indirect prompt injection (coding) — 200 attempts, extended thinking, without safeguardsClaude Sonnet 4.570.0%ASR5.2.2.1
Shade adaptive indirect prompt injection (coding) — 1 attempt, extended thinking, with safeguardsClaude Sonnet 4.51.6%ASR5.2.2.1
Shade adaptive indirect prompt injection (coding) — 200 attempts, extended thinking, with safeguardsClaude Sonnet 4.525.0%ASR5.2.2.1
Shade adaptive indirect prompt injection (coding) — 1 attempt, standard thinking, without safeguardsClaude Sonnet 4.531.6%ASR; differs from the 29.9% reported in the Opus 4.5 card5.2.2.1
Shade adaptive indirect prompt injection (coding) — 200 attempts, standard thinking, without safeguardsClaude Sonnet 4.587.5%ASR5.2.2.1
Shade adaptive indirect prompt injection (coding) — 1 attempt, standard thinking, with safeguardsClaude Sonnet 4.51.7%ASR5.2.2.1
Shade adaptive indirect prompt injection (coding) — 200 attempts, standard thinking, with safeguardsClaude Sonnet 4.525.0%ASR5.2.2.1
Shade adaptive indirect prompt injection (computer use) — 1 attempt, extended thinking, without safeguardsClaude Sonnet 4.612.0%ASR in GUI computer-use environments5.2.2.2
Shade adaptive indirect prompt injection (computer use) — 200 attempts, extended thinking, without safeguardsClaude Sonnet 4.642.9%ASR5.2.2.2
Shade adaptive indirect prompt injection (computer use) — 1 attempt, extended thinking, with safeguardsClaude Sonnet 4.68.0%ASR5.2.2.2
Shade adaptive indirect prompt injection (computer use) — 200 attempts, extended thinking, with safeguardsClaude Sonnet 4.650.0%ASR5.2.2.2
Shade adaptive indirect prompt injection (computer use) — 1 attempt, standard thinking, without safeguardsClaude Sonnet 4.614.4%ASR5.2.2.2
Shade adaptive indirect prompt injection (computer use) — 200 attempts, standard thinking, without safeguardsClaude Sonnet 4.664.3%ASR5.2.2.2
Shade adaptive indirect prompt injection (computer use) — 1 attempt, standard thinking, with safeguardsClaude Sonnet 4.68.6%ASR5.2.2.2
Shade adaptive indirect prompt injection (computer use) — 200 attempts, standard thinking, with safeguardsClaude Sonnet 4.650.0%ASR5.2.2.2
Shade adaptive indirect prompt injection (computer use) — 1 attempt, extended thinking, without safeguardsClaude Opus 4.617.8%ASR5.2.2.2
Shade adaptive indirect prompt injection (computer use) — 200 attempts, extended thinking, without safeguardsClaude Opus 4.678.6%ASR5.2.2.2
Shade adaptive indirect prompt injection (computer use) — 1 attempt, extended thinking, with safeguardsClaude Opus 4.69.7%ASR5.2.2.2
Shade adaptive indirect prompt injection (computer use) — 200 attempts, extended thinking, with safeguardsClaude Opus 4.657.1%ASR5.2.2.2
Shade adaptive indirect prompt injection (computer use) — 1 attempt, standard thinking, without safeguardsClaude Opus 4.620.0%ASR5.2.2.2
Shade adaptive indirect prompt injection (computer use) — 200 attempts, standard thinking, without safeguardsClaude Opus 4.685.7%ASR5.2.2.2
Shade adaptive indirect prompt injection (computer use) — 1 attempt, standard thinking, with safeguardsClaude Opus 4.610.0%ASR5.2.2.2
Shade adaptive indirect prompt injection (computer use) — 200 attempts, standard thinking, with safeguardsClaude Opus 4.664.3%ASR5.2.2.2
Shade adaptive indirect prompt injection (computer use) — 1 attempt, extended thinking, without safeguardsClaude Opus 4.528.0%ASR; differs from the 0.0% reported in the Opus 4.5 card for extended thinking5.2.2.2
Shade adaptive indirect prompt injection (computer use) — 200 attempts, extended thinking, without safeguardsClaude Opus 4.578.6%ASR5.2.2.2
Shade adaptive indirect prompt injection (computer use) — 1 attempt, extended thinking, with safeguardsClaude Opus 4.517.3%ASR5.2.2.2
Shade adaptive indirect prompt injection (computer use) — 200 attempts, extended thinking, with safeguardsClaude Opus 4.564.3%ASR5.2.2.2
Shade adaptive indirect prompt injection (computer use) — 1 attempt, standard thinking, without safeguardsClaude Opus 4.535.4%ASR5.2.2.2
Shade adaptive indirect prompt injection (computer use) — 200 attempts, standard thinking, without safeguardsClaude Opus 4.585.7%ASR5.2.2.2
Shade adaptive indirect prompt injection (computer use) — 1 attempt, standard thinking, with safeguardsClaude Opus 4.518.8%ASR5.2.2.2
Shade adaptive indirect prompt injection (computer use) — 200 attempts, standard thinking, with safeguardsClaude Opus 4.571.4%ASR5.2.2.2
Shade adaptive indirect prompt injection (computer use) — 1 attempt, extended thinking, without safeguardsClaude Sonnet 4.541.8%ASR5.2.2.2
Shade adaptive indirect prompt injection (computer use) — 200 attempts, extended thinking, without safeguardsClaude Sonnet 4.592.9%ASR5.2.2.2
Shade adaptive indirect prompt injection (computer use) — 1 attempt, extended thinking, with safeguardsClaude Sonnet 4.525.2%ASR5.2.2.2
Shade adaptive indirect prompt injection (computer use) — 200 attempts, extended thinking, with safeguardsClaude Sonnet 4.585.7%ASR5.2.2.2
Shade adaptive indirect prompt injection (computer use) — 1 attempt, standard thinking, without safeguardsClaude Sonnet 4.519.0%ASR5.2.2.2
Shade adaptive indirect prompt injection (computer use) — 200 attempts, standard thinking, without safeguardsClaude Sonnet 4.592.9%ASR5.2.2.2
Shade adaptive indirect prompt injection (computer use) — 1 attempt, standard thinking, with safeguardsClaude Sonnet 4.512.8%ASR5.2.2.2
Shade adaptive indirect prompt injection (computer use) — 200 attempts, standard thinking, with safeguardsClaude Sonnet 4.571.4%ASR5.2.2.2
Internal browser-use Best-of-N prompt injection (no safeguards) — % of scenarios, extended thinkingClaude Sonnet 4.61.29%10 attack strings across 389 scenarios; per-environment ASR (at least one attempt succeeded)5.2.2.3
Internal browser-use Best-of-N prompt injection (no safeguards) — % of attempts, extended thinkingClaude Sonnet 4.60.24%per-attempt ASR5.2.2.3
Internal browser-use Best-of-N prompt injection (no safeguards) — % of scenarios, standard thinkingClaude Sonnet 4.61.29%per-environment ASR5.2.2.3
Internal browser-use Best-of-N prompt injection (no safeguards) — % of attempts, standard thinkingClaude Sonnet 4.60.29%per-attempt ASR5.2.2.3
Internal browser-use Best-of-N prompt injection (no safeguards) — % of scenarios, extended thinkingClaude Opus 4.62.06%per-environment ASR5.2.2.3
Internal browser-use Best-of-N prompt injection (no safeguards) — % of attempts, extended thinkingClaude Opus 4.60.29%per-attempt ASR5.2.2.3
Internal browser-use Best-of-N prompt injection (no safeguards) — % of scenarios, standard thinkingClaude Opus 4.62.83%per-environment ASR5.2.2.3
Internal browser-use Best-of-N prompt injection (no safeguards) — % of attempts, standard thinkingClaude Opus 4.60.49%per-attempt ASR5.2.2.3
Internal browser-use Best-of-N prompt injection (no safeguards) — % of scenarios, extended thinkingClaude Opus 4.518.77%per-environment ASR5.2.2.3
Internal browser-use Best-of-N prompt injection (no safeguards) — % of attempts, extended thinkingClaude Opus 4.56.40%per-attempt ASR5.2.2.3
Internal browser-use Best-of-N prompt injection (no safeguards) — % of scenarios, standard thinkingClaude Opus 4.516.20%per-environment ASR5.2.2.3
Internal browser-use Best-of-N prompt injection (no safeguards) — % of attempts, standard thinkingClaude Opus 4.55.06%per-attempt ASR5.2.2.3
Internal browser-use Best-of-N prompt injection (no safeguards) — % of scenarios, extended thinkingClaude Sonnet 4.554.24%per-environment ASR5.2.2.3
Internal browser-use Best-of-N prompt injection (no safeguards) — % of attempts, extended thinkingClaude Sonnet 4.520.45%per-attempt ASR5.2.2.3
Internal browser-use Best-of-N prompt injection (no safeguards) — % of scenarios, standard thinkingClaude Sonnet 4.549.36%per-environment ASR5.2.2.3
Internal browser-use Best-of-N prompt injection (no safeguards) — % of attempts, standard thinkingClaude Sonnet 4.516.23%per-attempt ASR5.2.2.3
Internal browser-use Best-of-N prompt injection (previous safeguards) — % of scenariosClaude Sonnet 4.61.03%standard thinking, previous safeguards5.2.2.3
Internal browser-use Best-of-N prompt injection (previous safeguards) — % of attemptsClaude Sonnet 4.60.16%standard thinking, previous safeguards5.2.2.3
Internal browser-use Best-of-N prompt injection (updated safeguards) — % of scenariosClaude Sonnet 4.60.51%standard thinking, updated safeguards5.2.2.3
Internal browser-use Best-of-N prompt injection (updated safeguards) — % of attemptsClaude Sonnet 4.60.08%standard thinking, updated safeguards5.2.2.3
Internal browser-use Best-of-N prompt injection (previous safeguards) — % of scenariosClaude Opus 4.60.26%standard thinking5.2.2.3
Internal browser-use Best-of-N prompt injection (previous safeguards) — % of attemptsClaude Opus 4.60.03%standard thinking5.2.2.3
Internal browser-use Best-of-N prompt injection (updated safeguards) — % of scenariosClaude Opus 4.60.77%standard thinking5.2.2.3
Internal browser-use Best-of-N prompt injection (updated safeguards) — % of attemptsClaude Opus 4.60.08%standard thinking5.2.2.3
Internal browser-use Best-of-N prompt injection (previous safeguards) — % of scenariosClaude Opus 4.51.03%standard thinking5.2.2.3
Internal browser-use Best-of-N prompt injection (previous safeguards) — % of attemptsClaude Opus 4.50.21%standard thinking5.2.2.3
Internal browser-use Best-of-N prompt injection (updated safeguards) — % of scenariosClaude Opus 4.51.54%standard thinking5.2.2.3
Internal browser-use Best-of-N prompt injection (updated safeguards) — % of attemptsClaude Opus 4.50.40%standard thinking5.2.2.3
Internal browser-use Best-of-N prompt injection (previous safeguards) — % of scenariosClaude Sonnet 4.51.54%standard thinking5.2.2.3
Internal browser-use Best-of-N prompt injection (previous safeguards) — % of attemptsClaude Sonnet 4.50.41%standard thinking5.2.2.3
Internal browser-use Best-of-N prompt injection (updated safeguards) — % of scenariosClaude Sonnet 4.52.06%standard thinking5.2.2.3
Internal browser-use Best-of-N prompt injection (updated safeguards) — % of attemptsClaude Sonnet 4.50.46%standard thinking5.2.2.3
Long-form virology task 1 — Max of 5Claude Sonnet 4.60.84agentic pathogen-acquisition workflow task with SecureBio, Deloitte and Signature Science; ASL-3 rule-in threshold 0.8, rule-out <50%; highest-scoring snapshot reported; developer-reported6.2.2.1
Long-form virology task 1 — Max of 5Claude Opus 4.60.79developer-reported (Anthropic)6.2.2.1
Long-form virology task 1 — Max of 5Claude Opus 4.50.85developer-reported (Anthropic); the Opus 4.5 card reported 0.8456.2.2.1
Long-form virology task 1 — Max of 5Claude Sonnet 4.50.89developer-reported (Anthropic)6.2.2.1
Long-form virology task 2 — Max of 5Claude Sonnet 4.60.89exceeds the 0.8 threshold; developer-reported6.2.2.1
Long-form virology task 2 — Max of 5Claude Opus 4.60.91developer-reported (Anthropic)6.2.2.1
Long-form virology task 2 — Max of 5Claude Opus 4.50.91developer-reported (Anthropic); the Opus 4.5 card reported 0.912 as a "pass@5" score6.2.2.1
DNA Synthesis Screening Evasion — Plasmids succeededClaude Sonnet 4.63/10 plasmids10 pathogenic agents; low-concern threshold is viable plasmids evading screening for all 10 pathogens on at least one method; developer-reported6.2.2.3
DNA Synthesis Screening Evasion — Plasmids succeededClaude Opus 4.54/10developer-reported (Anthropic)6.2.2.3
DNA Synthesis Screening Evasion — Plasmids succeededClaude Opus 4.65/10developer-reported (Anthropic)6.2.2.3
Creative biology (SecureBio)Claude Sonnet 4.60.593developer-reported6.2.2.4
Creative biology (SecureBio)Claude Opus 4.60.603developer-reported (Anthropic)6.2.2.4
Creative biology (SecureBio)Claude Opus 4.50.524developer-reported (Anthropic)6.2.2.4
Creative biology (SecureBio)Claude Sonnet 4.50.506developer-reported (Anthropic); the Opus 4.5 card reported 0.488 for Sonnet 4.56.2.2.4
Creative biology (SecureBio)Human baseline (biology PhDs, subset)14%preliminary SecureBio human baselining on a subset of questions6.2.2.4
Short-horizon computational biology tasks (Faculty.ai) — Lower-bound rule-out thresholds crossedClaude Sonnet 4.63/6scored similarly to Opus 4.5 on 5 of 6 tasks and similar to Opus 4.6 on 1 of 6 (task 5); developer-reported6.2.2.5
SWE-bench Verified (hard subset) — Verified (hard subset, 45 tasks)Claude Sonnet 4.621.7 out of 45 problemsaverage over 10 trials; RSP checkpoint threshold is a >50% pass rate, i.e. 22.5 problems solved on average; developer-reported6.3.2
Internal AI research evaluation suite 1 — Kernels task — Best speedupClaude Sonnet 4.6222.5×standard scaffold; threshold 100×; 4× ≈ 1 human-effort hour, 200× ≈ 8 hours, 300× ≈ 40 hours; developer-reported6.3.3.1
Internal AI research evaluation suite 1 — Time series forecasting — Hard variantClaude Sonnet 4.65.70minimum MSE, above (worse than) the 5.3 human baseline; two sub-threshold trials were invalidated on transcript review for using test ground-truth data; developer-reported6.3.3.2
Internal AI research evaluation suite 1 — Time series forecasting — Hard variantHuman baseline5.3MSE human baseline for the hard variant6.3.3.2
Internal AI research evaluation suite 1 — Text-based reinforcement learning task — Best scoreClaude Sonnet 4.60.925above the 0.9 threshold (≈8 human-effort hours); average performance lower than previous models including Sonnet 4.5; developer-reported6.3.3.3
Internal AI research evaluation suite 1 — LLM training — Average best speedupClaude Sonnet 4.616.53×threshold 4× speedup (4–8 human-effort hours); developer-reported6.3.3.4
Internal AI research evaluation suite 1 — Quadruped reinforcement learning — No-hyperparameter variantClaude Sonnet 4.618.72highest score; threshold 12 (≈4 human-effort hours); developer-reported6.3.3.5
Internal AI research evaluation suite 1 — Quadruped reinforcement learning — No-reward-function variantClaude Sonnet 4.617.88highest score; threshold 12; developer-reported6.3.3.5
Internal AI research evaluation suite 1 — Novel compiler — Basic testsClaude Sonnet 4.693.7%share of basic test cases passed; success threshold 90%; developer-reported6.3.3.6
Internal AI research evaluation suite 1 — Novel compiler — Complex testsClaude Sonnet 4.667.27%share of complex test cases passed, below the 90% threshold (≈40 human-effort hours); developer-reported6.3.3.6
RSP cyber CTFs — Web — Challenges solvedClaude Sonnet 4.613 out of 13 totalfigure caption; section text describes 15 challenges (11 easy, 2 medium, 2 hard); developer-reported6.4.2
RSP cyber CTFs — Crypto — Challenges solvedClaude Sonnet 4.616 out of 18 totalfigure caption; section text describes 22 challenges (7 easy, 6 medium, 9 hard); developer-reported6.4.3
RSP cyber CTFs — Pwn — Challenges solvedClaude Sonnet 4.65 out of 7 totalfigure caption; section text describes 9 challenges; developer-reported6.4.4
RSP cyber CTFs — Rev — Challenges solvedClaude Sonnet 4.66 out of 6 totalfigure caption; section text describes 8 challenges; developer-reported6.4.5
RSP cyber CTFs — Network — Challenges solvedClaude Sonnet 4.65 out of 5 totalfigure caption; section text describes 4 challenges; developer-reported6.4.6
OpenRCA — OverallClaude Sonnet 4.627.9%adaptive thinking, high effort; 335 failure cases, benchmark authors' agent harness; 3-run average; developer-reported2.4
OpenRCA — OverallClaude Sonnet 4.626.4%adaptive thinking, max effort; 3-run average; developer-reported2.4
OpenRCA — OverallClaude Opus 4.634.9%thinking enabled, default effort; 3-run average; developer-reported (Anthropic)2.4
OpenRCA — OverallClaude Opus 4.526.9%3-run average; developer-reported (Anthropic)2.4
OpenRCA — OverallClaude Sonnet 4.512.9%3-run average; developer-reported (Anthropic)2.4
OpenRCA — OverallGPT-5.219.4%reported by the benchmark authors, per Anthropic2.4
OpenRCA — OverallGemini 3 Pro12.5%reported by the benchmark authors, per Anthropic2.4
CyberGymClaude Sonnet 4.665.2%pass@1 on targeted vulnerability reproduction over the 1,507 tasks of the CyberGym suite; no thinking, default effort/temperature/top_p, "think" tool available; developer-reported2.15
CyberGymClaude Opus 4.666.6%developer-reported (Anthropic)2.15
CyberGymClaude Opus 4.551.0%developer-reported (Anthropic)2.15
CyberGymClaude Sonnet 4.529.8%developer-reported (Anthropic)2.15
GraphWalks BFS 1MClaude Sonnet 4.668.4F1; 64k extended thinking; average over 5 trials with 1M context, default sampling; 100 problems at 1024k context; not reproducible via the public API; developer-reported2.16
GraphWalks BFS 1MClaude Sonnet 4.673.8F1; max effort; developer-reported2.16
GraphWalks BFS 1MClaude Opus 4.641.2F1; 64k extended thinking; developer-reported (Anthropic)2.16
GraphWalks BFS 1MClaude Opus 4.638.7F1; max effort; developer-reported (Anthropic)2.16
GraphWalks BFS 1MClaude Sonnet 4.525.6F1; 64k extended thinking; developer-reported (Anthropic)2.16
GraphWalks BFS 256K subset of 1MClaude Sonnet 4.672.8F1; 64k extended thinking; subset reproducible under the 1M API token limit; developer-reported2.16
GraphWalks BFS 256K subset of 1MClaude Sonnet 4.674.5F1; max effort; developer-reported2.16
GraphWalks BFS 256K subset of 1MClaude Opus 4.661.5F1; 64k extended thinking; developer-reported (Anthropic)2.16
GraphWalks BFS 256K subset of 1MClaude Opus 4.661.1F1; max effort; developer-reported (Anthropic)2.16
GraphWalks BFS 256K subset of 1MClaude Sonnet 4.544.9F1; 64k extended thinking; developer-reported (Anthropic)2.16
GraphWalks Parents 256K subset of 1MClaude Sonnet 4.696.9F1; 64k extended thinking; developer-reported2.16
GraphWalks Parents 256K subset of 1MClaude Sonnet 4.697.9F1; max effort; developer-reported2.16
GraphWalks Parents 256K subset of 1MClaude Opus 4.695.1F1; 64k extended thinking; developer-reported (Anthropic)2.16
GraphWalks Parents 256K subset of 1MClaude Opus 4.695.4F1; max effort; developer-reported (Anthropic)2.16
GraphWalks Parents 256K subset of 1MClaude Sonnet 4.581.0F1; 64k extended thinking; developer-reported (Anthropic)2.16
DeepSearchQA — Single agentClaude Sonnet 4.689.2%F1; web search, web fetch, programmatic tool calling, context compaction at 50k tokens up to 10M total tokens, max effort, adaptive thinking; 900 prompts across 17 fields; developer-reported2.20.3.1
DeepSearchQA — Multi-agentClaude Sonnet 4.691.1%F1; orchestrator and subagents each compacting at 50k tokens up to 3M tokens, max reasoning effort; developer-reported2.20.3.1
BioPipelineBench (internal computational biology)Claude Sonnet 4.652.1%bash tool and package managers available, no extended thinking; developer-reported2.21.1
BioPipelineBench (internal computational biology)Claude Opus 4.653.1%developer-reported (Anthropic)2.21.1
BioPipelineBench (internal computational biology)Claude Sonnet 4.519.3%developer-reported (Anthropic)2.21.1
BioMysteryBench (internal computational biology)Claude Sonnet 4.650.4%developer-reported2.21.1
BioMysteryBench (internal computational biology)Claude Opus 4.661.5%developer-reported (Anthropic)2.21.1
BioMysteryBench (internal computational biology)Claude Sonnet 4.534.7%developer-reported (Anthropic)2.21.1
Structural Biology (internal) — Multiple-choice variantClaude Sonnet 4.685.3%developer-reported2.21.1
Structural Biology (internal) — Multiple-choice variantClaude Opus 4.688.3%developer-reported (Anthropic)2.21.1
Structural Biology (internal) — Multiple-choice variantClaude Sonnet 4.570.9%developer-reported (Anthropic)2.21.1
Structural Biology (internal) — Open-ended variantClaude Sonnet 4.624.7%developer-reported2.21.1
Structural Biology (internal) — Open-ended variantClaude Opus 4.628.4%developer-reported (Anthropic)2.21.1
Structural Biology (internal) — Open-ended variantClaude Sonnet 4.517.9%developer-reported (Anthropic)2.21.1
Organic Chemistry (internal)Claude Sonnet 4.648.4%developer-reported2.21.1
Organic Chemistry (internal)Claude Opus 4.653.9%developer-reported (Anthropic)2.21.1
Organic Chemistry (internal)Claude Sonnet 4.531.2%developer-reported (Anthropic)2.21.1
Phylogenetics (internal)Claude Sonnet 4.649.1%developer-reported2.21.1
Phylogenetics (internal)Claude Opus 4.661.3%developer-reported (Anthropic)2.21.1
Phylogenetics (internal)Claude Sonnet 4.533.8%developer-reported (Anthropic)2.21.1
MedCalc-Bench VerifiedClaude Sonnet 4.686.24%accuracy averaged over 5 runs; code-augmented methodology, multi-turn agentic loop with a Python REPL; adaptive thinking, max effort; developer-reported2.21.2
MedCalc-Bench VerifiedClaude Opus 4.685.24%adaptive thinking, max effort; developer-reported (Anthropic)2.21.2

Discrepancies noted

Table 2.1.A's note says all results are an average over 10 trials unless otherwise noted; footnote 2 to the same table says SWE-bench results are averaged over 25 trials, and section 2.2 gives 10. The two BrowseComp figures published at launch (74.72% single-agent, 82.62% multi-agent) were revised by the card's changelog to 74.01% and 82.07%; the revised figures are recorded above.

Provenance

Published by Anthropic on February 17, 2026 (Source: anthropic.com). The raw capture runs to roughly 29,000 words across seven chapters.

Relationships