"Training Verifiers to Solve Math Word Problems" is a paper by Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Łukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse and John Schulman of OpenAI, posted to arXiv as 2110.14168 in October 2021 (Cobbe and Kosaraju are marked as equal contributors). It introduces the GSM8K dataset of grade-school math word problems and proposes training verifiers that rank sampled solutions at test time. The dataset is released at github.com/openai/grade-school-math.
Summary
The paper opens from the observation that large language models, despite scaling trends that hold across many orders of magnitude, "falter when required to perform multi-step mathematical reasoning," and that autoregressive models have no mechanism to correct their own errors, so that a solution which veers off course becomes unrecoverable. Extrapolating from generative methods alone, the authors argue, would require "an exorbitant parameter count to achieve even moderate performance" on distributions such as the MATH dataset. This motivates two contributions: a dataset of moderate difficulty on which progress can be measured, and a verification method with what the authors describe as more favourable scaling.
Dataset
GSM8K consists of 8.5K grade-school math problems created by human problem writers, split into 7.5K training and 1K test problems. Problems take between 2 and 8 steps to solve, using a sequence of elementary calculations with basic arithmetic operations; the authors state that "a bright middle school student should be able to solve every problem," that no concepts beyond early algebra are required, and that the vast majority of problems can be solved without explicitly defining a variable. Four design principles are stated: high quality (human-written rather than scraped, with agreement-based quality control), high diversity (problems not drawn from a shared linguistic template), moderate difficulty (challenging for large models "without being completely intractable"), and natural-language solutions, which the authors expect to "shed light on the properties of large language models' internal monologues."
Appendix A describes collection: an initial set of a thousand problems from freelance contractors hired on Upwork, then scaled collection through Surge AI, an NLP data-labelling platform. Contractors were given seed questions generated by a few-shot-prompted 175B GPT-3 model, which they could use directly, adapt, or ignore, and pairwise similarity scores were computed to discourage template reuse. After collection, workers re-solved all problems without seeing their own; disagreements were repaired or discarded. A second agreement check on a smaller subset found 1.7% of problems still producing disagreement, which the authors give as their estimate of the fraction with breaking errors or ambiguities, adding that "a larger percentage of problems contain subtle errors" is possible.
The related-work section positions GSM8K against ASDiv (2.3K problems, sharing GSM8K's diversity and quality principles but smaller, without natural-language solutions and with fewer steps per problem) and against the MATH dataset, which is "larger and significantly more complex than GSM8K" but whose difficulty "makes it challenging to accurately measure progress."
Methods and results
Two methods are compared on GPT-3 models, mainly at 6B and 175B parameters. The baseline is fine-tuning with the language-modelling objective and a single low-temperature sample at test time. Verification samples multiple high-temperature solutions, scores each with a verifier trained to predict whether a solution reaches the correct final answer, and returns the highest-ranked one. All models are trained to use a calculator through calculation annotations injected into the training set, because "models frequently fail to accurately perform calculations."
Results are presented as charts. In the text, the authors report that the 175B model "significantly outperforms" smaller models when fine-tuned, and that a log-linear extrapolation implies a model of 10^16 parameters would be needed to reach an 80% solve rate on the full training set; along the data dimension, the 175B model "would require at least two additional orders of magnitude of training data" for the same target. Fine-tuning a 6B model to output the final answer directly without intermediate steps drops performance from 20.6% to 5.2%.
The verifier pipeline fine-tunes a generator for 2 epochs, samples 100 completions per training problem labelled correct or incorrect by final answer, and trains the verifier for a single epoch, with a language-modelling auxiliary objective. At test time 100 completions per problem are ranked. The authors find verification is not beneficial at small dataset sizes but gives "a strong boost" once the dataset is large enough, with 175B verifiers "taking off" earlier than 6B ones. The conclusion states that on the full dataset "6B verification slightly outperforms a finetuned 175B model, thereby offering a boost approximately equivalent to a 30x model size increase," and that verifiers "scale significantly better with increased data."
Ablations find that token-level verifiers (predicting correctness after every token) ultimately outperform solution-level verifiers and are less prone to overfitting; that including the language-modelling objective is "a strict improvement"; and that a large generator with a small verifier performs "significantly better" than the reverse, which the authors take to suggest the verifier "may often be relying on relatively coarse heuristics." At test time, 6B verifier performance improves with the number of completions up to 400 and then declines, which the authors attribute to "the risk of finding adversarial solutions that fool the verifier." Majority voting among the top verifier-ranked solutions helps further; with 100 samples the top 3–5 should vote, with 3,200 samples roughly the top 30. Residual dropout at 20% improves both fine-tuning and verification.
Key claims
- GSM8K: 8.5K human-written grade-school word problems, 7.5K train and 1K test, 2–8 solution steps, basic arithmetic only, natural-language solutions with calculator annotations.
- Estimated breaking-error or ambiguity rate: 1.7% of problems, from contractor disagreement on a re-solve check; subtle errors may be more common.
- Verification (100 samples ranked by a trained verifier) with a 6B model slightly outperforms fine-tuning a 175B model; the authors equate this to a roughly 30x model-size increase.
- Removing intermediate reasoning from a fine-tuned 6B model cuts test accuracy from 20.6% to 5.2%.
- Naive log-linear extrapolation of fine-tuning results implies 10^16 parameters for an 80% solve rate.
- Verifier gains reverse beyond about 400 samples per problem; dropout of 20% is a strong regulariser for both methods.
- The paper reports no human accuracy figure; its numerical results are otherwise given as charts rather than tables.
Provenance
Retrieved 22 September 2026 from the arXiv HTML rendering of version 2 (arxiv.org); the abstract page is arxiv.org. The acknowledgements thank Dan Hendrycks, Leo Gao, Alec Radford and Giambattista Parascandolo for feedback and "the team at Surge AI" for data collection.
Relationships
- supports: GSM8K — the primary text behind the benchmark's description, construction and original results.
- related: Measuring Mathematical Problem Solving With the MATH Dataset (Hendrycks et al., 2021) — the harder contemporaneous dataset the paper positions GSM8K against.
- related: Let's Verify Step by Step (Lightman et al., OpenAI, 2023) — OpenAI's later extension of the verifier approach from outcome to process supervision.
- contradicts: GSM-Symbolic: Understanding the Limitations of Mathematical Reasoning in Large Language Models (Mirzadeh et al., Apple, 2024) — later work arguing that GSM8K scores overstate reasoning ability.
- related: A Careful Examination of Large Language Model Performance on Grade School Arithmetic (Zhang et al., Scale AI, 2024) — a 2024 held-out replication testing contamination of GSM8K.
- instance-of: AI Benchmarks and Evaluation