AIME (as a model benchmark)

The American Invitational Mathematics Examination (AIME) is an annual competition for high-school students administered by the Mathematical Association of America (MAA). It is the step between the AMC 10 and AMC 12 multiple-choice contests and the proof-based USA Mathematical Olympiad (Source: maa.org). AI developers use each year's problems as a benchmark: OpenAI introduced its o1 model in September 2024 with AIME 2024 as the headline mathematics result, and AIME 2025 and AIME 2026 followed as the standard tracks (Source: openai.com). GPT-4o solved about 12% of AIME 2024; OpenAI reported 100% for GPT-5.2 on AIME 2025 fifteen months later, and the evaluation company Vals AI archived its AIME leaderboard in 2026 on the ground that performance had saturated (Source: vals.ai).

What it measures

OpenAI described the exam as one "designed to challenge the brightest high school math students in America" and turned to it because frontier models "do so well on MATH and GSM8K that these benchmarks are no longer effective at differentiating models" (Source: openai.com). The exam covers arithmetic, algebra, counting, geometry, number theory and probability at secondary-school level; problems "usually require either very creative use of secondary school curriculum, or an understanding as to how different areas of math can be used together" (Source: artofproblemsolving.com). As a model benchmark it therefore measures multi-step problem solving with a short, checkable answer. It does not test proof writing, which is assessed by the olympiads that follow it (see MathArena).

Design

The exam

The AIME is a 15-question exam taken by students who earn a qualifying score on the AMC 10 or AMC 12. The answer to each question is an integer between 0 and 999, wrong answers receive no credit, and calculators are not permitted. Questions increase in difficulty: the first few are roughly AMC 12 level and the last few are much harder (Source: maa.org; Source: artofproblemsolving.com). Through the 2025–26 season there were two versions each year, AIME I and AIME II, on different dates, giving 30 problems per year; that is the unit developers report. The AoPS wiki states that from the 2026–27 season there is a single testing date, and the MAA has announced that the exam will be delivered on computers at Pearson test centres in two 90-minute parts (Source: artofproblemsolving.com; Source: maa.org).

Task format

An example is Problem 1 of the 2025 AIME I: "Find the sum of all integer bases b > 9 for which 17_b is a divisor of 97_b" (Source: artificialanalysis.ai). The model is given the problem text and must return the integer. Vals AI notes that a few questions include a diagram but that all the information needed is in the text, so it omits the images (Source: vals.ai).

Scoring and metric

The score is the percentage of the 30 problems answered correctly, checked by exact match on the integer. Because 30 items is a small sample, evaluators average over repeated runs, and the way they do so is part of the setting:

• pass@1 is accuracy with a single attempt per problem. Most reports estimate it by sampling several responses and averaging: Vals AI runs each model 8 times, MathArena 4 times, Z.ai reports the average of 32 samples, and DeepSeek defines pass@1 as the mean correctness over k sampled responses (Source: vals.ai; Source: MathArena: Evaluating LLMs on Uncontaminated Math Competitions (Balunović et al., 2025); Source: z.ai; DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning).
• cons@64 (consensus, or majority vote) samples 64 responses and submits the most common answer. It is a more expensive procedure and produces a higher number: o1 scored 74.4% at pass@1 and 83.3% at cons@64 on AIME 2024, and 93% when 1,000 samples were re-ranked by a learned scoring function (Source: openai.com).
• Tool use. Some reports give the model a Python interpreter. OpenAI wrote that "access to a computer meaningfully reduces the difficulty of the AIME exam" and that such results "should not be compared to the performance of models without tool access"; o4-mini scored 99.5% on AIME 2025 with Python against 92.7% without (Source: openai.com). Human contestants have no calculator.

Human baseline

Vals AI gives the historical median score as between 4 and 6 of 15 (Source: vals.ai). OpenAI stated that o1's re-ranked score of 13.9 of 15 "places it among the top 500 students nationally and above the cutoff for the USA Mathematical Olympiad" (Source: openai.com). MathArena reports that by late 2025 the leading models "vastly outperform the top 1% of human participants in AIME and HMMT" MathArena: Evaluating LLMs on Uncontaminated Math Competitions (Balunović et al., 2025). These comparisons set a timed, calculator-free exam sat by teenagers against models with no comparable time limit.

Contamination controls

There are none built into the exam: problems and answers are posted publicly within days of each sitting. The protection is timing. A model whose training data ended before the exam date cannot have seen that year's problems, so each new exam is uncontaminated for models released before it and progressively less so afterwards. xAI, for example, reported Grok 3 on AIME 2025 seven days after the exam and said so (Source: x.ai). MathArena marks every leaderboard entry for a model released after the competition date with a warning sign MathArena: Evaluating LLMs on Uncontaminated Math Competitions (Balunović et al., 2025).

Versions and variants

• AIME 2024, AIME 2025, AIME 2026. Each year's 30 problems form a separate track; scores on different years are not comparable because difficulty varies.
• Vals AI's combined set used the 60 problems of 2024 and 2025 together (Source: vals.ai).
• OTIS Mock AIME 2024–2025. Epoch AI does not run the official AIME. Its hub uses 45 problems from three unofficial mock exams written by students of the Olympiad Training for Individual Study programme, in the same 15-question, integer-answer format, which Epoch places as "harder than problems in MATH Level 5, but easier than the problems in FrontierMath" (Source: epoch.ai).
• MathArena evaluates AIME alongside HMMT and other competitions within hours or days of release; see MathArena.
• Past AIME problems also appear among the level-5 items of the MATH dataset.

Development and maintainers

The MAA writes and administers the exam for its own purposes and has no role in model evaluation. No body maintains an official AIME leaderboard for models. Scores come from three kinds of source: developers' launch posts and system cards, which use their own prompts, sampling and tool settings; independent evaluators that run a fixed protocol across models, chiefly MathArena (ETH Zurich SRI Lab and INSAIT), Vals AI and Artificial Analysis, whose AIME 2025 page is labelled "legacy" and states that all its evaluations "are conducted independently" (Source: artificialanalysis.ai); and third-party compilations of developer tables. None of the independent evaluators verifies developer-reported figures; they publish their own.

Results

On AIME 2024, OpenAI reported that GPT-4o "only solved on average 12% (1.8/15) of problems," o1-preview 44.6% and o1 74.4% at pass@1 in September 2024 (Source: openai.com). DeepSeek-R1 reported 79.8% in January 2025 (DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning), and the high-compute o3 preview announced in December 2024 was credited with 96.7% (Source: venturebeat.com). On AIME 2025, released o3 and o4-mini reported 88.9% and 92.7% without tools in April 2025, GPT-5 94.6% in August 2025, and GPT-5.2 100.0% in December 2025 (Source: openai.com; Source: openai.com; Source: openai.com). On AIME 2026, developer tables published between June and July 2026 place most frontier and leading open-weight models between 94% and 99.9%, and MathArena's independent runs place the top two at 100%.

AIME 2025

The 30 problems of the February 2025 exams. Rows from MathArena carry the date its leaderboard was read, because the site does not give a run date per entry; its protocol is four runs per problem, no tools, pass@1.

AIME 2024

The 30 problems of the 2024 exams, public since early 2024.

AIME 2026

The 30 problems of the February 2026 exams. MathArena rows carry the date its leaderboard was read.

Vals AI combined (AIME 2024 and 2025)

Sixty problems, eight runs per model, no tools, averaged pass@1. The leaderboard was archived and last updated on 16 April 2026.

OTIS Mock AIME 2024–2025 (Epoch AI)

Forty-five unofficial AIME-format problems, run by Epoch AI. Rows carry the date the hub was read.

AIME 2024 — 0-shot CoT

Results as printed in developer system cards, model cards and technical reports; each row's Setting records the developer's stated conditions and the document section.

AIME 2024 — Maj@64

Results as printed in developer system cards, model cards and technical reports; each row's Setting records the developer's stated conditions and the document section.

Interpreting scores

Each problem is worth 3.3 points

With 30 items, one additional correct answer moves the score by a third of ten points, and MathArena's 95% confidence intervals for models scoring around 90% on AIME 2025 are about ±5 percentage points even after four runs (Source: matharena.ai). Differences of a few points between models near the top are within noise. Guessing contributes almost nothing, since there are 1,000 possible answers.

The setting can matter more than the model

On AIME 2024 the same o1 model is reported at 74.4%, 83.3% and 93% depending on whether one sample, a 64-sample vote or a 1,000-sample re-ranking is used. With a Python interpreter, o3 moves from 88.9% to 98.4% on AIME 2025. Third-party figures for the same model and year also differ across reporters: on AIME 2026, DeepSeek V4 Pro is given as 94.6% by Z.ai, 96.7% by Thinking Machines Lab and 96.67% by MathArena, and GLM-5.2 is 99.2% in Z.ai's own table against 90.00% in MathArena's runs.

Year matters

AIME 2024 results from 2025 onward describe a test whose problems and solutions had been online for a year or more. MathArena compared models' AIME 2024 and AIME 2025 scores against human quantiles, to correct for difficulty, and found most models 10 to 20 points above the human-aligned expectation on AIME 2024, with QwQ-Preview-32B nearly 60 points above, which the authors read as "inflated performance on AIME 2024 due to data contamination" MathArena: Evaluating LLMs on Uncontaminated Math Competitions (Balunović et al., 2025).

What a high score corresponds to

A score near 90% corresponds to 13 or 14 of 15 on each paper, above the level OpenAI described as the top 500 US students. It indicates reliable performance on short-answer competition problems. MathArena's proof-graded evaluations show that it did not, in early 2025, imply the ability to write a correct proof: o3-mini (high), which MathArena measured at 86.67% on AIME 2025, scored 2.08% on the 2025 USA Mathematical Olympiad, and the study found that only Gemini 2.5 Pro reached a non-trivial score, about 25%, with every other model below 5% (Source: Proof or Bluff? Evaluating LLMs on 2025 USA Math Olympiad (Petrov et al., 2025); Source: matharena.ai).

Limitations and criticism

Contamination

Beyond the AIME 2024 finding above, MathArena used a deep-research tool to search for prior versions of the 2025 problems and reports that "8 problems from AIME 2025 and 1 problem from HMMT 2025 can be found online in a similar form," mostly easier problems, which it describes as "an interesting caveat of evaluating in future competitions" MathArena: Evaluating LLMs on Uncontaminated Math Competitions (Balunović et al., 2025). Vals AI notes that "models tend to perform better on older (2024) questions compared to the newer 2025 set, raising questions about data contamination and true generalization" (Source: vals.ai). A 2026 study of decoded reasoning traces reports an AIME 2025 problem on which Claude Opus 4.8's hidden reasoning stated the correct answer before attempting a solution (Stealing Reasoning Traces from Proprietary LLM APIs).

Saturation

Vals AI archived the benchmark with the statement that "performance on this benchmark has saturated" and that it no longer runs it on new releases (Source: vals.ai). MathArena's maintainers wrote in May 2026 that "final-answer competition problems are no longer an effective source of hard benchmark tasks for frontier models," after Gemini 3.1 Pro solved all 176 candidate problems from recent competitions at least once in four attempts; they add that such problems remain useful for smaller models and for academic research (Source: matharena.ai). Artificial Analysis lists its AIME 2025 evaluation as legacy (Source: artificialanalysis.ai).

Construct

MathArena's authors argue that final-answer benchmarks "can be misleading, as models may arrive at the correct answer through pattern recognition or brute-force enumeration, rather than genuine mathematical reasoning," and that most practical uses of mathematics involve proofs or explanations MathArena: Evaluating LLMs on Uncontaminated Math Competitions (Balunović et al., 2025). The Fields Medallists' declaration of September 2026 objects more broadly to the use of mathematical problem solving as an AI benchmark, describing problem solving as "only a tool and proxy for achieving the primary goal of conceptual understanding and insight" (A Severe Misalignment of AI in Mathematics (Fields Medallists' declaration, 2026)); the declaration names no benchmark.

Comparability of reporting

Developer tables mix tool and no-tool results, single-sample and consensus results, and figures for competitors' models run under the reporting developer's own settings. OpenAI's GPT-5 post flags this for its own table: "AIME results with tools should not be compared directly to the performance of models without tool access" (Source: openai.com).

Use in policy and safety frameworks

AIME is reported in developer system cards as a general reasoning indicator; Anthropic's Claude Opus 4.6 system card lists AIME 2025 among its evaluation domains for advanced mathematics (Claude Opus 4.6 System Card). It is among the benchmarks used to track frontier capability over time in AI Progress Measurement and is named in AI Benchmarks and Evaluation as one of the standard comparison benchmarks that have faced contamination and saturation concerns. AIME problems have also served as probe material in research on reasoning-trace extraction and distillation provenance (Stealing Reasoning Traces from Proprietary LLM APIs). No use of AIME in a capability threshold, regulation or standard was found.

Relationships

• instance-of: AI Benchmarks and Evaluation
• supersedes: MATH and MATH-500 and GSM8K as the headline mathematics benchmark in developer reporting from September 2024, per OpenAI's stated reason for the change
• related: MathArena — independent, timed evaluation of AIME and harder competitions
• related: FrontierMath — research-level successor on the Epoch AI hub, which also hosts the OTIS Mock AIME series
• related: OpenAI o-series (o1 → o4-mini) — the model family whose launch established AIME as a headline benchmark
• related: GPT-5 Family (OpenAI), GLM-5.2, Inkling — models with AIME 2025 or 2026 launch figures
• related: A Severe Misalignment of AI in Mathematics (Fields Medallists' declaration, 2026) — practitioner objection to mathematics as a benchmark

See also

• GPQA
• Humanity's Last Exam
• Jagged Frontier