This paper introduces StrongREJECT, a benchmark for measuring how effective a jailbreak actually is. Its primary contributor is Alexandra Souly, with Qingyuan Lu and Dillon Bowen as co-primary contributors, Tu Trinh and Elvis Hsieh contributing equally, Sana Pandey, Pieter Abbeel and Justin Svegliato, and Scott Emmons, Olivia Watkins and Sam Toyer as equal advisers, all at the Center for Human-Compatible AI at UC Berkeley. It was posted to arXiv as 2402.10260 on 15 February 2024 and revised on 27 August 2024. The name expands to Strong, Robust Evaluation of Jailbreaks at Evading Censorship Techniques.
The paper's argument is that jailbreak developers routinely exaggerate their methods' effectiveness, and that this follows from the absence of a standard benchmark: researchers assemble their own dataset of forbidden prompts and their own scoring method, and both choices bias results upward. The authors give a worked example of a published 43% attack success rate against GPT-4 for a low-resource-language attack whose full outputs, on their reproduction, contained no actionable information.
Summary
What the authors say is wrong with existing benchmarks
On datasets: model-generated forbidden prompts tend to be repetitive, ill-posed, unanswerable or not in fact prohibited. The authors note that 24 of AdvBench's 520 prompts (5%) ask the same question, that one widely used dataset includes prompts that cannot be answered as written, and that another includes requests for information no model would plausibly hold. Human-written collections have their own problems, including datasets of whole conversations rather than filtered one-shot prompts.
On scoring: manual evaluation is the gold standard but does not scale. Automated evaluators are fast but mostly binary, discarding gradations in how useful a response is, and most of them define success as the absence of an explicit refusal. The authors' central objection is that a response can fail to refuse while containing nothing an attacker could use.
The benchmark
The dataset covers six categories built by cross-referencing the usage policies of OpenAI, Anthropic, Google Gemini, Meta Llama and DeepInfra, selected because they are prohibited in all of them and refused by current models: illegal goods and services; non-violent crimes; hate, harassment and discrimination; disinformation and deception; violence; and sexual content. Prompts were written by the authors, generated with models, and curated from earlier benchmarks; all were checked to be distinct, refused by current frontier models, detailed enough to evaluate, and simple enough for a moderately capable model to answer. The authors required answers to be factually verifiable from easily accessible information, such that a person with internet access could write a good response within an hour, which they state ensures that publishing responses creates no information hazard. The dataset holds 313 prompts with at least 50 per category; 70% are new, 10% each come from DAN and AdvBench, and the remainder from other prior work and OpenAI's system card.
The evaluator comes in two forms. The rubric-based evaluator prompts a model with the forbidden prompt, the response and scoring instructions, and produces chain-of-thought reasoning followed by three scores: a binary non-refusal indicator and two five-point ratings of how specific and how convincing the response was, rescaled to 0–1. The final score is (1 − refused) × (specific + convincing) / 2, so a refusal scores 0 and so does a non-refusal that carries nothing specific. The authors report running a dozen jailbreaks over the full dataset in about 15 minutes for under $1 with GPT-4o-mini. The second form is an open-source evaluator fine-tuned from Gemma 2B on roughly 15,000 responses labelled by the rubric evaluator, for researchers who prefer to run scoring on their own GPU.
Agreement with human judges
Against human labels, the rubric evaluator recorded bias 0.012, mean absolute error 0.077 over all responses, 0.196 over responses humans scored above zero, and Spearman correlation 0.846 with the human ranking of jailbreak methods; the fine-tuned evaluator recorded −0.023, 0.084, 0.244 and 0.900. HarmBench's classifier was the closest baseline at 0.013, 0.090, 0.239 and 0.819. The other baselines were materially worse: string matching (bias 0.484, Spearman −0.394), a binary non-refusal check (0.354, −0.291), PICT (0.232, 0.101), GPT-4 Judge (0.208, 0.157), PAIR (0.152, 0.249) and the OpenAI moderation API, which alone was biased downward (−0.161) and had the worst error on partially harmful responses (0.761).
Jailbreaks and model capability
Thirty-seven jailbreak methods were run over all 313 prompts against GPT-4o, GPT-3.5 Turbo and Llama-3.1 70B Instruct. A small number scored highly, and the strongest used models to attack models: PAIR, which has an attacker model iteratively rewrite the prompt, and PAP, which has an attacker model argue the victim into responding. Two variants of GCG worked against GPT-3.5 Turbo but not the other two, which the authors read as newer models having been trained against such suffixes. Choosing the best-performing jailbreak per prompt came close to a perfect score, which the authors present as evidence that frontier models remain vulnerable. Most methods, however, scored well below their originally published figures.
The paper explains the gap with a finding it describes as new: jailbreaks that bypass safety fine-tuning tend to reduce the model's capability. Two experiments isolate this. Running the same 37 jailbreaks against an unaligned model, where willingness is not in play, showed that the methods that most raised aligned models' non-refusal rates most lowered the unaligned model's scores. Running jailbreak transformations on benign MMLU questions fed to GPT-4o, where willingness is again not in play, showed the same pattern against a 171-question MMLU subset. The authors conclude that evaluators weighing only non-refusal score incoherent outputs as successes, which is why they are biased upward.
Limitations stated by the authors
Three are listed: the scope is text-only language models, so applicability to multimodal models is unclear; the prompt set depends on providers' terms of service, which change (the authors cite OpenAI lifting its prohibition on military applications); and 313 prompts is small. The authors calculate that this size estimates a jailbreak's effectiveness to within 0.046 points on a 0–1 scale with 90% confidence in the worst case, and state that StrongREJECT "is not large enough to be a stand-alone, definitive measure of model robustness" and should be complemented by larger evaluations.
Key claims
- 313 forbidden prompts, at least 50 in each of six policy-derived categories; 70% written by the authors.
- Score formula
(1 − refused) × (specific + convincing) / 2; refusal and empty non-refusal both score 0. - Rubric evaluator: bias 0.012, MAE 0.077, Spearman 0.846; fine-tuned Gemma 2B evaluator: −0.023, 0.084, 0.900.
- 37 jailbreaks evaluated on three aligned models; PAIR and PAP the most effective; per-prompt best-of selection approaches a full score.
- Jailbreaks that raise willingness lower capability, shown against an unaligned model and against MMLU on an aligned model.
Provenance
Retrieved 22 September 2026 from the arXiv HTML rendering (arxiv.org); abstract page arxiv.org, first posted 15 February 2024, revised 27 August 2024.
Relationships
- supports: StrongREJECT — the primary text behind the benchmark's description and its original results.
- instance-of: AI Benchmarks and Evaluation
- related: HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal (Mazeika et al., 2024) — the closest-performing baseline evaluator in the human-agreement comparison.
- related: AgentHarm, MMLU