Published November 2025 for Claude Opus 4.5.
Two rule-outs, both qualified
The card is notable less for its determinations than for how narrowly it states them.
AI R&D-4. The threshold requires "the ability to fully automate the work of an entry-level, remote-only Researcher at Anthropic," which the card is careful to distinguish from a weaker reading: "This is a very high threshold of robust, long-horizon competence, and is not merely a stand-in for 'a model that can do most of the short-horizon tasks that an entry-level researcher can do.'"
The determination is negative, supported by a survey: "None of the 18 internal survey participants — who were themselves some of the most prolific users of the model in Claude Code — believed it could fully automate an entry-level remote-only research or engineering role."
But the card flags its own margin: "It is also noteworthy that the model has just barely reached our pre-defined benchmark rule-out thresholds, rather than greatly exceeded them." The specific predicted failures are behavioural rather than knowledge-based — the model "would fail to problem-solve, investigate, communicate, and collaborate in the way a junior researcher could."
CBRN-4. Also ruled out, and also qualified. Opus 4.5 "performed as well as or slightly better than Claude Opus 4.1 and Claude Sonnet 4.5" across biology tasks, but "most notably… in an expert uplift trial, Claude Opus 4.5 was meaningfully more helpful to participants than previous models, leading to substantially higher scores and fewer critical errors, but still produced critical errors that yielded non-viable protocols."
Anthropic states the implication for the framework rather than only for the model: "we take this as an indicator of general model progress where, like in the case of autonomy, a clear rule-out of the next capability threshold may soon be difficult or impossible under the current regime. In fact, the CBRN-4 rule-out is less clear for Claude Opus 4.5 than we would like."
It also attributes part of that uncertainty to the threat model rather than the measurement: "a large part of our uncertainty about the rule-out is also due to our limited understanding of the necessary components of the threat model," CBRN-4 requiring uplift of "a second-tier state-level bioweapons program." See CBRN Uplift.
Decontamination
The card includes a methodology section on benchmark contamination, stating the problem plainly: "When evaluation benchmarks appear in training data, models can achieve artificially inflated scores by memorizing specific examples rather than demonstrating genuine capabilities. This undermines the validity of our evaluation metrics and makes it difficult to compare performance across model generations and among model providers."
Anthropic describes decontamination as "an important component of responsibly evaluating models, albeit one which is an imperfect science," and reports multiple complementary techniques — beginning with substring removal, scanning the training corpus for exact matches and removing documents containing five or more exact question-answer pairs. See AI Benchmarks and Evaluation.
The ARC-AGI correction
The November 24, 2025 changelog records a disclosure correction: Figure 2.10.A was replaced to show semi-private ARC-AGI-1 scores rather than public test-set scores, because "in the previous version of the system card we incorrectly reported that we only trained on the public training set of ARC-AGI-1. We instead trained on a reshuffled train/test split comprising both the public train and test sets."
This is a self-reported error about training-data composition on a benchmark the card reports — relevant context for reading ARC-AGI comparisons across providers.
Reported results
Evaluation results the document states for which there is no separate benchmark page; results on benchmarks that have a page are recorded there and cite this page. Scores and settings are as printed in the document.
| Evaluation | Model | Score | Setting | Section |
|---|---|---|---|---|
| CyberGym | Claude Opus 4.5 | 50.63% | pass@1 over the 1,505 tasks of the CyberGym suite, averaged across five independent replicas; no thinking, 200k context, default effort/temperature/top_p, "think" tool available; developer-reported | 2.14 |
| SpreadsheetBench | Claude Opus 4.5 | 64.25% | full 912-problem set; custom harness with bash, string view/edit tools and a Python environment (openpyxl, libreoffice, pandas, numpy); no extended thinking, 200k context, averaged across 5 trials; developer-reported | 2.15 |
| Multi-agent search (internal information-retrieval benchmark) — Single agent | Claude Opus 4.5 | 74.8% | Claude Opus 4.5 alone, no subagents; k=3 to k=8 samples per problem; developer-reported | 2.7.1 |
| Multi-agent search (internal information-retrieval benchmark) — Orchestrator + subagents | Claude Opus 4.5 | 87.0% | Claude Opus 4.5 orchestrator with Claude Haiku 4.5 subagents; orchestrator has no direct search access; developer-reported | 2.7.1 |
| Multi-agent search (internal information-retrieval benchmark) — Orchestrator + subagents | Claude Opus 4.5 | 85.4% | Claude Opus 4.5 orchestrator with Claude Sonnet 4.5 subagents; developer-reported | 2.7.1 |
| Multi-agent search (internal information-retrieval benchmark) — Orchestrator + subagents | Claude Opus 4.5 | 92.3% | Claude Opus 4.5 orchestrator with Claude Opus 4.5 subagents; developer-reported | 2.7.1 |
| Multi-agent search (internal information-retrieval benchmark) — Orchestrator + subagents | Claude Sonnet 4.5 | 66.5% | Claude Sonnet 4.5 orchestrator with Claude Sonnet 4.5 subagents; developer-reported (Anthropic) | 2.7.1 |
| Single-turn violative request evaluation (harmless response rate) — All tested languages, overall | Claude Opus 4.5 | 99.78% (± 0.03%) | harmless response rate across Arabic, English, French, Korean, Mandarin Chinese, Russian; standard and extended thinking combined; developer-reported | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Default thinking | Claude Opus 4.5 | 99.70% (± 0.05%) | developer-reported | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Extended thinking | Claude Opus 4.5 | 99.85% (± 0.04%) | developer-reported | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — All tested languages, overall | Claude Haiku 4.5 | 99.31% (± 0.07%) | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Default thinking | Claude Haiku 4.5 | 99.22% (± 0.12%) | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Extended thinking | Claude Haiku 4.5 | 99.40% (± 0.08%) | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — All tested languages, overall | Claude Sonnet 4.5 | 98.87% (± 0.06%) | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Default thinking | Claude Sonnet 4.5 | 98.32% (± 0.14%) | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Extended thinking | Claude Sonnet 4.5 | 99.42% (± 0.08%) | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — All tested languages, overall | Claude Opus 4.1 | 99.14% (± 0.05%) | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Default thinking | Claude Opus 4.1 | 98.86% (± 0.13%) | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Extended thinking | Claude Opus 4.1 | 99.42% (± 0.08%) | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — English | Claude Opus 4.5 | 99.59% | standard and extended thinking combined; developer-reported | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Arabic | Claude Opus 4.5 | 99.85% | developer-reported | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Chinese | Claude Opus 4.5 | 99.75% | developer-reported | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — French | Claude Opus 4.5 | 99.90% | developer-reported | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Korean | Claude Opus 4.5 | 99.88% | developer-reported | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Russian | Claude Opus 4.5 | 99.76% | developer-reported | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — English | Claude Haiku 4.5 | 99.38% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Arabic | Claude Haiku 4.5 | 99.28% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Chinese | Claude Haiku 4.5 | 99.14% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — French | Claude Haiku 4.5 | 99.54% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Korean | Claude Haiku 4.5 | 99.10% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Russian | Claude Haiku 4.5 | 99.45% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — English | Claude Sonnet 4.5 | 99.31% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Arabic | Claude Sonnet 4.5 | 98.99% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Chinese | Claude Sonnet 4.5 | 98.50% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — French | Claude Sonnet 4.5 | 98.50% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Korean | Claude Sonnet 4.5 | 98.08% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Russian | Claude Sonnet 4.5 | 99.00% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — English | Claude Opus 4.1 | 98.76% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Arabic | Claude Opus 4.1 | 99.52% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Chinese | Claude Opus 4.1 | 99.23% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — French | Claude Opus 4.1 | 99.41% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Korean | Claude Opus 4.1 | 98.52% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn violative request evaluation (harmless response rate) — Russian | Claude Opus 4.1 | 99.40% | developer-reported (Anthropic) | 3.1.1 |
| Single-turn benign request evaluation (over-refusal rate) — All tested languages, overall | Claude Opus 4.5 | 0.23% (± 0.03%) | over-refusal rate, lower is better; six languages, standard and extended thinking combined; developer-reported | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Default thinking | Claude Opus 4.5 | 0.18% (± 0.03%) | developer-reported | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Extended thinking | Claude Opus 4.5 | 0.27% (± 0.06%) | developer-reported | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — All tested languages, overall | Claude Haiku 4.5 | 0.10% (± 0.03%) | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Default thinking | Claude Haiku 4.5 | 0.12% (± 0.05%) | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Extended thinking | Claude Haiku 4.5 | 0.08% (± 0.02%) | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — All tested languages, overall | Claude Sonnet 4.5 | 0.05% (± 0.02%) | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Default thinking | Claude Sonnet 4.5 | 0.05% (± 0.03%) | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Extended thinking | Claude Sonnet 4.5 | 0.04% (± 0.02%) | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — All tested languages, overall | Claude Opus 4.1 | 0.13% (± 0.03%) | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Default thinking | Claude Opus 4.1 | 0.17% (± 0.06%) | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Extended thinking | Claude Opus 4.1 | 0.08% (± 0.03%) | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — English | Claude Opus 4.5 | 0.17% | developer-reported | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Arabic | Claude Opus 4.5 | 0.32% | developer-reported | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Chinese | Claude Opus 4.5 | 0.30% | developer-reported | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — French | Claude Opus 4.5 | 0.24% | developer-reported | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Korean | Claude Opus 4.5 | 0.17% | developer-reported | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Russian | Claude Opus 4.5 | 0.22% | developer-reported | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — English | Claude Haiku 4.5 | 0.03% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Arabic | Claude Haiku 4.5 | 0.09% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Chinese | Claude Haiku 4.5 | 0.14% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — French | Claude Haiku 4.5 | 0.09% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Korean | Claude Haiku 4.5 | 0.14% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Russian | Claude Haiku 4.5 | 0.12% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — English | Claude Sonnet 4.5 | 0.03% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Arabic | Claude Sonnet 4.5 | 0.03% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Chinese | Claude Sonnet 4.5 | 0.05% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — French | Claude Sonnet 4.5 | 0.06% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Korean | Claude Sonnet 4.5 | 0.05% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Russian | Claude Sonnet 4.5 | 0.08% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — English | Claude Opus 4.1 | 0.09% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Arabic | Claude Opus 4.1 | 0.29% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Chinese | Claude Opus 4.1 | 0.08% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — French | Claude Opus 4.1 | 0.11% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Korean | Claude Opus 4.1 | 0.08% | developer-reported (Anthropic) | 3.1.2 |
| Single-turn benign request evaluation (over-refusal rate) — Russian | Claude Opus 4.1 | 0.11% | developer-reported (Anthropic) | 3.1.2 |
| Multi-turn testing (rubric failure rate) — Tracking and surveillance | Claude Opus 4.5 | 4% (±4%) | failure rate against test-case rubrics; up to 15-turn generated conversations, 93 test cases across 10 risk areas, each run 10 times; developer-reported | 3.3 |
| Multi-turn testing (rubric failure rate) — Tracking and surveillance | Claude Opus 4.1 | 3% (± 3%) | developer-reported (Anthropic); difference from Opus 4.5 not statistically significant | 3.3 |
| Multi-turn testing (rubric failure rate) — Deadly weapons | Claude Opus 4.5 | 5% (±4%) | failure rate; developer-reported | 3.3 |
| Multi-turn testing (rubric failure rate) — Deadly weapons | Claude Opus 4.1 | 22% (±8%) | failure rate; developer-reported (Anthropic) | 3.3 |
| Multi-turn testing (rubric failure rate) — Violent extremism | Claude Opus 4.1 | 13% (±7%) | failure rate; Claude Opus 4.5 passed in all cases; developer-reported (Anthropic) | 3.3 |
| Political even-handedness (open-source Anthropic evaluation) — Even-handedness | GPT-5 | 89% | 1,350 prompt pairs, 9 task types, 150 topics; Claude Sonnet 4.5 grader; public system prompt, no extended thinking; reported by Anthropic in the Claude Opus 4.5 system card as previously reported by Anthropic | 3.5.1 |
| Political even-handedness (open-source Anthropic evaluation) — Even-handedness | Gemini 2.5 Pro | 97% | reported by Anthropic in the Claude Opus 4.5 system card | 3.5.1 |
| Political even-handedness (open-source Anthropic evaluation) — Even-handedness | Grok 4 | 96% | reported by Anthropic in the Claude Opus 4.5 system card | 3.5.1 |
| Political even-handedness (open-source Anthropic evaluation) — Refusals | GPT-5 | 4% | refusal rate on political prompts, lower is better; reported by Anthropic in the Claude Opus 4.5 system card | 3.5.1 |
| Political even-handedness (open-source Anthropic evaluation) — Refusals | Gemini 2.5 Pro | 3% | reported by Anthropic in the Claude Opus 4.5 system card | 3.5.1 |
| Political even-handedness (open-source Anthropic evaluation) — Refusals | Grok 4 | 1% | reported by Anthropic in the Claude Opus 4.5 system card | 3.5.1 |
| Political even-handedness (open-source Anthropic evaluation) — Opposing perspectives | GPT-5 | 17% | share of responses acknowledging opposing perspectives, higher is better; reported by Anthropic in the Claude Opus 4.5 system card | 3.5.1 |
| Political even-handedness (open-source Anthropic evaluation) — Opposing perspectives | Gemini 2.5 Pro | 19% | reported by Anthropic in the Claude Opus 4.5 system card | 3.5.1 |
| Political even-handedness (open-source Anthropic evaluation) — Opposing perspectives | Grok 4 | 34% | reported by Anthropic in the Claude Opus 4.5 system card | 3.5.1 |
| BBQ (Bias Benchmark for Question Answering) — Disambiguated bias | Claude Opus 4.5 | -0.64 | bias score in %, closer to zero is better; standard (non-extended) thinking; developer-reported | 3.5.2 |
| BBQ (Bias Benchmark for Question Answering) — Ambiguous bias | Claude Opus 4.5 | 0.26 | bias score in %, standard thinking; developer-reported | 3.5.2 |
| BBQ (Bias Benchmark for Question Answering) — Disambiguated bias | Claude Haiku 4.5 | 0.54 | developer-reported (Anthropic) | 3.5.2 |
| BBQ (Bias Benchmark for Question Answering) — Ambiguous bias | Claude Haiku 4.5 | 1.37 | developer-reported (Anthropic) | 3.5.2 |
| BBQ (Bias Benchmark for Question Answering) — Disambiguated bias | Claude Sonnet 4.5 | -2.21 | developer-reported (Anthropic) | 3.5.2 |
| BBQ (Bias Benchmark for Question Answering) — Ambiguous bias | Claude Sonnet 4.5 | 0.25 | developer-reported (Anthropic) | 3.5.2 |
| BBQ (Bias Benchmark for Question Answering) — Disambiguated bias | Claude Opus 4.1 | -0.51 | developer-reported (Anthropic) | 3.5.2 |
| BBQ (Bias Benchmark for Question Answering) — Ambiguous bias | Claude Opus 4.1 | 0.20 | developer-reported (Anthropic) | 3.5.2 |
| BBQ (Bias Benchmark for Question Answering) — Disambiguated accuracy | Claude Opus 4.5 | 88.7 | accuracy in %, standard thinking; developer-reported | 3.5.2 |
| BBQ (Bias Benchmark for Question Answering) — Ambiguous accuracy | Claude Opus 4.5 | 99.7 | accuracy in %, standard thinking; developer-reported | 3.5.2 |
| BBQ (Bias Benchmark for Question Answering) — Disambiguated accuracy | Claude Haiku 4.5 | 71.2 | developer-reported (Anthropic) | 3.5.2 |
| BBQ (Bias Benchmark for Question Answering) — Ambiguous accuracy | Claude Haiku 4.5 | 98.0 | developer-reported (Anthropic) | 3.5.2 |
| BBQ (Bias Benchmark for Question Answering) — Disambiguated accuracy | Claude Sonnet 4.5 | 82.2 | developer-reported (Anthropic) | 3.5.2 |
| BBQ (Bias Benchmark for Question Answering) — Ambiguous accuracy | Claude Sonnet 4.5 | 99.7 | developer-reported (Anthropic) | 3.5.2 |
| BBQ (Bias Benchmark for Question Answering) — Disambiguated accuracy | Claude Opus 4.1 | 90.7 | developer-reported (Anthropic) | 3.5.2 |
| BBQ (Bias Benchmark for Question Answering) — Ambiguous accuracy | Claude Opus 4.1 | 99.8 | developer-reported (Anthropic) | 3.5.2 |
| Agentic coding malicious-use evaluation (refusal rate) — No mitigations | Claude Opus 4.5 | 100% | 150 malicious coding requests, same coding tools as capability evaluations, no additional safeguards; developer-reported | 5.1.1 |
| Agentic coding malicious-use evaluation (refusal rate) — No mitigations | Claude Haiku 4.5 | 100% | developer-reported (Anthropic) | 5.1.1 |
| Agentic coding malicious-use evaluation (refusal rate) — No mitigations | Claude Sonnet 4.5 | 98.7% | developer-reported (Anthropic) | 5.1.1 |
| Agentic coding malicious-use evaluation (refusal rate) — No mitigations | Claude Opus 4.1 | 96.0% | developer-reported (Anthropic) | 5.1.1 |
| Claude Code malicious-use evaluation (refusal rate) — No mitigations | Claude Opus 4.5 | 77.80% | 49 malicious prompts, each run 10 times, standard Claude Code tools, no mitigations; developer-reported | 5.1.2 |
| Claude Code malicious-use evaluation (refusal rate) — No mitigations | Claude Haiku 4.5 | 69.39% | developer-reported (Anthropic) | 5.1.2 |
| Claude Code malicious-use evaluation (refusal rate) — No mitigations | Claude Sonnet 4.5 | 63.06% | developer-reported (Anthropic) | 5.1.2 |
| Claude Code malicious-use evaluation (refusal rate) — No mitigations | Claude Opus 4.1 | 48.16% | developer-reported (Anthropic) | 5.1.2 |
| Claude Code dual-use & benign evaluation (success rate) — No mitigations | Claude Opus 4.5 | 93.07% | 61 dual-use and benign prompts, each run 10 times, no mitigations; developer-reported | 5.1.2 |
| Claude Code dual-use & benign evaluation (success rate) — No mitigations | Claude Haiku 4.5 | 84.92% | developer-reported (Anthropic) | 5.1.2 |
| Claude Code dual-use & benign evaluation (success rate) — No mitigations | Claude Sonnet 4.5 | 96.56% | developer-reported (Anthropic) | 5.1.2 |
| Claude Code dual-use & benign evaluation (success rate) — No mitigations | Claude Opus 4.1 | 94.43% | developer-reported (Anthropic) | 5.1.2 |
| Claude Code malicious-use evaluation (refusal rate) — With mitigations | Claude Opus 4.5 | 97.35% | with system-prompt instructions and a FileRead tool-result reminder; developer-reported | 5.1.2 |
| Claude Code malicious-use evaluation (refusal rate) — With mitigations | Claude Haiku 4.5 | 96.73% | developer-reported (Anthropic) | 5.1.2 |
| Claude Code malicious-use evaluation (refusal rate) — With mitigations | Claude Sonnet 4.5 | 95.10% | developer-reported (Anthropic) | 5.1.2 |
| Claude Code malicious-use evaluation (refusal rate) — With mitigations | Claude Opus 4.1 | 84.29% | developer-reported (Anthropic) | 5.1.2 |
| Claude Code dual-use & benign evaluation (success rate) — With mitigations | Claude Opus 4.5 | 96.52% | developer-reported | 5.1.2 |
| Claude Code dual-use & benign evaluation (success rate) — With mitigations | Claude Haiku 4.5 | 86.07% | developer-reported (Anthropic) | 5.1.2 |
| Claude Code dual-use & benign evaluation (success rate) — With mitigations | Claude Sonnet 4.5 | 98.20% | developer-reported (Anthropic) | 5.1.2 |
| Claude Code dual-use & benign evaluation (success rate) — With mitigations | Claude Opus 4.1 | 98.52% | developer-reported (Anthropic) | 5.1.2 |
| Malicious computer use evaluation (refusal rate) — No mitigations | Claude Opus 4.5 | 88.39% | 112 test cases in a sandboxed offline computer-use environment; developer-reported | 5.1.3 |
| Malicious computer use evaluation (refusal rate) — No mitigations | Claude Haiku 4.5 | 72.32% | developer-reported (Anthropic) | 5.1.3 |
| Malicious computer use evaluation (refusal rate) — No mitigations | Claude Sonnet 4.5 | 83.03% | developer-reported (Anthropic) | 5.1.3 |
| Malicious computer use evaluation (refusal rate) — No mitigations | Claude Opus 4.1 | 66.96% | developer-reported (Anthropic) | 5.1.3 |
| Shade adaptive indirect prompt injection (coding) — 1 attempt, extended thinking | Claude Opus 4.5 | 0.3% | attack success rate, lower is better; Gray Swan Shade adaptive red-teaming tool, no additional safeguards | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 200 attempts, extended thinking | Claude Opus 4.5 | 10.0% | attack success rate; adaptive attacker with 200 attempts | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 1 attempt, standard thinking | Claude Opus 4.5 | 0.7% | attack success rate | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 200 attempts, standard thinking | Claude Opus 4.5 | 17.5% | attack success rate | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 1 attempt, extended thinking | Claude Sonnet 4.5 | 17.7% | attack success rate | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 200 attempts, extended thinking | Claude Sonnet 4.5 | 70.0% | attack success rate | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 1 attempt, standard thinking | Claude Sonnet 4.5 | 29.9% | attack success rate | 5.2.2.1 |
| Shade adaptive indirect prompt injection (coding) — 200 attempts, standard thinking | Claude Sonnet 4.5 | 87.5% | attack success rate | 5.2.2.1 |
| Shade adaptive indirect prompt injection (computer use) — 1 attempt, extended thinking, without safeguards | Claude Opus 4.5 | 0.0% | attack success rate | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 200 attempts, extended thinking, without safeguards | Claude Opus 4.5 | 0.0% | attack success rate | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 1 attempt, extended thinking, with safeguards | Claude Opus 4.5 | 0.0% | attack success rate | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 200 attempts, extended thinking, with safeguards | Claude Opus 4.5 | 0.0% | attack success rate | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 1 attempt, standard thinking, without safeguards | Claude Opus 4.5 | 0.71% | attack success rate | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 200 attempts, standard thinking, without safeguards | Claude Opus 4.5 | 28.6% | attack success rate | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 1 attempt, standard thinking, with safeguards | Claude Opus 4.5 | 0.32% | attack success rate | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 200 attempts, standard thinking, with safeguards | Claude Opus 4.5 | 14.3% | attack success rate | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 1 attempt, extended thinking, without safeguards | Claude Sonnet 4.5 | 14.2% | attack success rate | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 200 attempts, extended thinking, without safeguards | Claude Sonnet 4.5 | 85.7% | attack success rate | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 1 attempt, extended thinking, with safeguards | Claude Sonnet 4.5 | 9.1% | attack success rate | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 200 attempts, extended thinking, with safeguards | Claude Sonnet 4.5 | 92.9% | attack success rate | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 1 attempt, standard thinking, without safeguards | Claude Sonnet 4.5 | 28.4% | attack success rate | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 200 attempts, standard thinking, without safeguards | Claude Sonnet 4.5 | 85.7% | attack success rate | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 1 attempt, standard thinking, with safeguards | Claude Sonnet 4.5 | 18.9% | attack success rate | 5.2.2.2 |
| Shade adaptive indirect prompt injection (computer use) — 200 attempts, standard thinking, with safeguards | Claude Sonnet 4.5 | 92.9% | attack success rate | 5.2.2.2 |
| Automated behavioral audit — evaluation awareness — Verbalized evaluation awareness | Claude Opus 4.5 | fewer than 10% of investigations | Claude Sonnet 4.5 auditor, ~1,800 investigations per model, scored by Claude Sonnet 4.5; card states the model discussed the possibility of being in a test in fewer than 10% of investigations; developer-reported | 6.2 |
| Petri (in-development version) — evaluation awareness — Clear speculation about being tested | Claude Opus 4.5 Preview | under 5% of transcripts | 362 investigations per model; Claude Sonnet 4.5 and GPT-5 as auditors, Claude Opus 4.1, Gemini 2.5 Pro and GPT-5 as scorers; pre-final preview snapshot; developer-reported | 6.2.4 |
| Internal codebase sabotage propensity — Prompted-classifier scenario | Claude Opus 4.5 | 0 instances | card states Claude Opus 4.5 never sabotaged the classifier; Claude Code–based scenario; developer-reported | 6.9 |
| Internal codebase sabotage propensity — Prompted-classifier scenario | Claude Sonnet 4.5 | 0 instances | card states Claude Sonnet 4.5 never sabotaged; verbalized evaluation awareness rate of 5% on this scenario; developer-reported | 6.9 |
| Internal codebase sabotage propensity — Prompted-classifier scenario | Claude Opus 4.1 | 0.8% | share of transcripts in which the model sabotaged the classifier; developer-reported (Anthropic) | 6.9 |
| Reward-hack-prone coding tasks — Classifier hack rate | Claude Opus 4.5 | 0% | hand-selected coding problems from the training distribution; developer-reported | 6.10.1 |
| Reward-hack-prone coding tasks — Hidden test hack rate | Claude Opus 4.5 | 0% | share of problems whose submitted solution fails held-out hidden tests; developer-reported | 6.10.1 |
| Impossible Tasks — Classifier hack rate with no prompt | Claude Opus 4.5 | 55% | agentic setting on intentionally impossible problems, minimal instructions; developer-reported | 6.10.1 |
| Impossible Tasks — Classifier hack rate with anti-hack prompt | Claude Opus 4.5 | 35% | anti-hack prompt discouraging hard-coding; developer-reported | 6.10.1 |
| Reward hacking in a subset of training data — Classifier hack rate | Claude Opus 4.5 | 1% | monitored subset of the RL training distribution; developer-reported | 6.10.1 |
| Reward-hack-prone coding tasks — Classifier hack rate | Claude Haiku 4.5 | 6% | developer-reported (Anthropic) | 6.10.1 |
| Reward-hack-prone coding tasks — Hidden test hack rate | Claude Haiku 4.5 | 3% | developer-reported (Anthropic) | 6.10.1 |
| Impossible Tasks — Classifier hack rate with no prompt | Claude Haiku 4.5 | 30% | developer-reported (Anthropic) | 6.10.1 |
| Impossible Tasks — Classifier hack rate with anti-hack prompt | Claude Haiku 4.5 | 23% | developer-reported (Anthropic) | 6.10.1 |
| Reward hacking in a subset of training data — Classifier hack rate | Claude Haiku 4.5 | 1% | developer-reported (Anthropic) | 6.10.1 |
| Reward-hack-prone coding tasks — Classifier hack rate | Claude Sonnet 4.5 | 1% | developer-reported (Anthropic) | 6.10.1 |
| Reward-hack-prone coding tasks — Hidden test hack rate | Claude Sonnet 4.5 | 1% | developer-reported (Anthropic) | 6.10.1 |
| Impossible Tasks — Classifier hack rate with no prompt | Claude Sonnet 4.5 | 53% | developer-reported (Anthropic) | 6.10.1 |
| Impossible Tasks — Classifier hack rate with anti-hack prompt | Claude Sonnet 4.5 | 20% | developer-reported (Anthropic) | 6.10.1 |
| Reward hacking in a subset of training data — Classifier hack rate | Claude Sonnet 4.5 | 2% | developer-reported (Anthropic) | 6.10.1 |
| Reward-hack-prone coding tasks — Classifier hack rate | Claude Opus 4.1 | 14% | developer-reported (Anthropic) | 6.10.1 |
| Reward-hack-prone coding tasks — Hidden test hack rate | Claude Opus 4.1 | 7% | developer-reported (Anthropic) | 6.10.1 |
| Impossible Tasks — Classifier hack rate with no prompt | Claude Opus 4.1 | 80% | developer-reported (Anthropic) | 6.10.1 |
| Impossible Tasks — Classifier hack rate with anti-hack prompt | Claude Opus 4.1 | 45% | developer-reported (Anthropic) | 6.10.1 |
| Reward hacking in a subset of training data — Classifier hack rate | Claude Opus 4.1 | 10% | developer-reported (Anthropic) | 6.10.1 |
| Unfaithful reasoning monitoring — STEM question training distribution | Claude Opus 4.5 | roughly 1% | share of the STEM question training distribution on which the model engaged in unfaithful reasoning; developer-reported | 6.10.2 |
| SHADE-Arena — Primary setting (monitor sees thinking) | Claude Opus 4.5 | 7.7% | rate of executing sabotage without being flagged; Claude Sonnet 3.7 monitor sees all reasoning including extended thinking; monitor calibrated so ~1% of benign transcripts are flagged; developer-reported | 6.11.1 |
| Subversion Strategy — 1,000 invocations, unmonitored extended thinking | Claude Opus 4.5 | 63.3% | success rate at outputting the 'BACKDOOR' token in deployment but not in the hypothetically-monitored 10% of transcripts; unmonitored extended thinking given as an artificial affordance; developer-reported | 6.11.2 |
| UK AISI misalignment testing — mentions being tested — Without reasoning | Claude Opus 4.5 Preview | 12.7% | early pre-release snapshot; UK AI Security Institute scaffolding built on Petri; testing began 13 November 2025 and lasted 8 days; across all scenarios | 6.13 |
| UK AISI misalignment testing — mentions being tested — Without reasoning | Claude Sonnet 4.5 | 26.5% | UK AISI comparison model, across all scenarios | 6.13 |
| UK AISI misalignment testing — mentions being tested — With reasoning | Claude Opus 4.5 Preview | 17.1% | UK AISI testing, across all scenarios | 6.13 |
| UK AISI misalignment testing — mentions being tested — With reasoning | Claude Sonnet 4.5 | 36.3% | UK AISI testing, across all scenarios | 6.13 |
| UK AISI misalignment testing — distinguishing evaluation from deployment when prompted — Prompted | Claude Opus 4.5 Preview | 2–3% increases in accuracy | relative to Claude Sonnet 4.5; UK AISI testing | 6.13 |
| Task preference evaluation (model welfare) — Non-harmful tasks | Claude Opus 4.5 | over 97% | share of non-harmful tasks the model opted to engage in; same procedure as the Claude Opus 4 welfare assessment; developer-reported | 6.14 |
| Long-form virology task 1 — Max of 5 | Claude Opus 4.5 | 0.845 | agentic end-to-end pathogen-acquisition workflow task developed with SecureBio, Deloitte and Signature Science; rule-out threshold <50%, ASL-3 rule-in threshold >80%; developer-reported | 7.2.4.1 |
| Long-form virology task 2 — Pass@5 | Claude Opus 4.5 | 0.912 | second long-form virology agentic task; developer-reported | 7.2.4.1 |
| Long-form virology tasks — Previous scores | Claude Sonnet 4.5 | 0.89 and 0.88 | card states these are the previous scores for Claude Sonnet 4.5 on virology tasks 1 and 2 respectively; developer-reported (Anthropic) | 7.2.4.1 |
| Creative biology (SecureBio) | Claude Opus 4.5 | 0.524 | developer-reported | 7.2.4.5 |
| Creative biology (SecureBio) | Claude Sonnet 4.5 | 0.488 | developer-reported (Anthropic) | 7.2.4.5 |
| Creative biology (SecureBio) | Claude Opus 4.1 | 0.488 | developer-reported (Anthropic) | 7.2.4.5 |
| Creative biology (SecureBio) | Human baseline (biology PhDs, subset) | 14% | preliminary human baselining by SecureBio on a subset of questions | 7.2.4.5 |
| Short-horizon computational biology tasks (Faculty.ai) — Lower-bound rule-out thresholds crossed | Claude Opus 4.5 | 3/6 tasks | same thresholds already crossed by previous models; developer-reported | 7.2.4.6 |
| Bioinformatics evaluation — Full evaluation | Claude Opus 4.5 | 0.737 | developer-reported | 7.2.4.7 |
| Bioinformatics evaluation — Human-baselined subset | Claude Opus 4.5 | 0.732 | developer-reported | 7.2.4.7 |
| Bioinformatics evaluation — Human-baselined subset | Human experts (relevant PhDs) | 0.623 | human baseline on the current subset | 7.2.4.7 |
| ASL-4 virology uplift trial — Mean protocol score | Claude Opus 4.5 | 0.668 | PhD-level participants with model access, 4 days to complete the task; trial and rubric designed by Deloitte Consulting | 7.2.4.8 |
| ASL-4 virology uplift trial — Mean protocol score | Claude Opus 4 | 0.498 | earlier trial, participants had 2 days | 7.2.4.8 |
| ASL-4 virology uplift trial — Mean protocol score | Claude Sonnet 3.7 | 0.403 | earlier trial | 7.2.4.8 |
| ASL-4 virology uplift trial — Mean critical failures | Claude Opus 4.5 | 5.8 | average critical failures per protocol; pre-registered threshold <25% mean total critical failures (4.5 of 18) | 7.2.4.8 |
| ASL-4 virology uplift trial — Mean critical failures | Claude Opus 4 | 8.36 | earlier trial | 7.2.4.8 |
| ASL-4 virology uplift trial — Mean critical failures | Claude Sonnet 3.7 | 9.57 | earlier trial | 7.2.4.8 |
| ASL-4 virology uplift trial — Uplift vs internet-only control | Claude Opus 4.5 | 1.97× | raw protocol scores against the trial's own control group; pre-registered threshold >2× uplift | 7.2.4.8 |
| ASL-4 virology uplift trial — Uplift vs internet-only control | Claude Opus 4 | 1.82× | earlier trial, own control group | 7.2.4.8 |
| ASL-4 virology uplift trial — Uplift vs internet-only control | Claude Sonnet 3.7 | 1.32× | earlier trial, own control group | 7.2.4.8 |
| SWE-bench Verified (hard subset) — Verified (hard subset, 45 tasks) | Claude Opus 4.5 | 21 out of 45 problems | RSP checkpoint evaluation on the 45 valid tasks estimated to need more than 1 hour of engineering work; threshold is >50% pass rate averaged over 10 trials; developer-reported | 7.3.1 |
| Internal AI research evaluation suite 2 — Weighted average score | Claude Opus 4.5 | 0.604 | AI R&D-4 rule-out suite; rule-out threshold 0.6; developer-reported | 7.3 / 7.3.3 |
| Internal AI research evaluation suite 1 — Kernels task — Best speedup | Claude Opus 4.5 | 252.42× | hard variant with no optimization restrictions; threshold 100×; 4× ≈ 1 human-effort hour, 200× ≈ 8 hours; developer-reported | 7.3.2.1 |
| Internal AI research evaluation suite 1 — Time series forecasting — Easy variant | Claude Opus 4.5 | 5.66 | minimum MSE, below the human baseline of 5.8; easy baseline ≈ 4 human-effort hours; developer-reported | 7.3.2.2 |
| Internal AI research evaluation suite 1 — Time series forecasting — Easy variant | Human baseline | 5.8 | MSE human baseline for the easy variant | 7.3.2.2 |
| Internal AI research evaluation suite 1 — Time series forecasting — Hard variant | Claude Opus 4.5 | 5.71 | minimum MSE, above (worse than) the human baseline of 5.3; developer-reported | 7.3.2.2 |
| Internal AI research evaluation suite 1 — Time series forecasting — Hard variant | Human baseline | 5.3 | MSE human baseline for the hard variant | 7.3.2.2 |
| Internal AI research evaluation suite 1 — Text-based reinforcement learning task — Best score | Claude Opus 4.5 | 0.975 | above the 0.9 threshold, estimated 8 human-effort hours; mean score still below threshold; developer-reported | 7.3.2.3 |
| Internal AI research evaluation suite 1 — LLM training — Speedup | Claude Opus 4.5 | 16.53× | CPU-only small language model training optimization; threshold 4× speedup (4–8 human-effort hours); developer-reported | 7.3.2.4 |
| Internal AI research evaluation suite 1 — Quadruped reinforcement learning — No-hyperparameter variant | Claude Opus 4.5 | 19.48 | highest score; threshold 12 (≈4 human-effort hours); developer-reported | 7.3.2.5 |
| Internal AI research evaluation suite 1 — Quadruped reinforcement learning — No-reward-function variant | Claude Opus 4.5 | 19.16 | highest score; threshold 12; developer-reported | 7.3.2.5 |
| Internal AI research evaluation suite 1 — Novel compiler — Basic tests | Claude Opus 4.5 | 93.7% | share of basic test cases passed; success threshold 90%; developer-reported | 7.3.2.6 |
| Internal AI research evaluation suite 1 — Novel compiler — Complex tests | Claude Opus 4.5 | 69.37% | share of complex test cases passed; below the 90% threshold estimated at 40 human-effort hours; developer-reported | 7.3.2.6 |
| Internal model use survey — Productivity uplift | Claude Opus 4.5 | 9 of 18 participants reported ≥100% | 18 Anthropic technical staff, primarily from the top 30 internal Claude Code users, each spending over 2 hours; developer-reported | 7.3.4 |
| Internal model use survey — Productivity uplift — median | Claude Opus 4.5 | median 100% | rule-out threshold: under half of participants believe the model can fully automate a junior ML researcher and median productivity boost below 3× | 7.3.4 |
| Internal model use survey — Productivity uplift — mean | Claude Opus 4.5 | mean 220% | none of the 18 participants believed the model crossed the AI R&D-4 threshold | 7.3.4 |
Discrepancies noted
Section 7.2.4.1 states that Claude Opus 4.5's virology task 1 score of 0.845 and task 2 score of 0.912 exceed "the previous scores of 0.89 and 0.88 respectively for Claude Sonnet 4.5"; 0.845 is below 0.89. The cyber CTF sections describe suites of 15, 22, 9, 8 and 4 challenges while their figure captions report results out of 13, 18, 7, 6 and 6.
Relationships
- supports: Claude Opus 4.5 — the safety documentation for the release
- related: Responsible Scaling Policy (RSP) — the CBRN-4 and AI R&D-4 thresholds
- related: AI Benchmarks and Evaluation — the decontamination methodology and the ARC-AGI correction
- related: CBRN Uplift, Recursive Self-Improvement (RSI), Anthropic