Magistral (Mistral AI, June 2025)

"Magistral" is a technical paper by Mistral AI, submitted to arXiv on June 12, 2025, two days after the company announced the Magistral reasoning models (Source: https://arxiv.org/abs/2506.10910). It introduces Magistral Medium, trained from Mistral Medium 3 by reinforcement learning alone, and Magistral Small, a 24-billion-parameter model released with open weights under the Apache 2.0 license, and describes the company's reinforcement-learning stack, which the authors say was built from the ground up on Mistral's own models and infrastructure instead of on traces distilled from earlier reasoning models.

Method

The training algorithm is a modified form of Group Relative Policy Optimization. The reward combines checks on output formatting, correctness (verified final answers for mathematics, compiled and tested programs for code), a length penalty, and a language-consistency reward intended to keep the chain of thought in the user's language. The paper describes an asynchronous distributed infrastructure in which generators run continuously while trainers update weights, and a data pipeline that filters mathematics and code problems by format and by difficulty.

Models and results

Magistral Medium was trained from Mistral Medium 3 Instruct with no supervised fine-tuning on reasoning traces, in stages that increased data difficulty and permitted completion length. The paper reports that its reinforcement-learning pipeline alone "yields a nearly 50% accuracy increase in AIME '24 (pass@1), and 30% on LiveCodeBench (v5)" over the base model, and compares the results with DeepSeek-V3, DeepSeek-R1-Zero and DeepSeek-R1 figures taken from DeepSeek's paper. Magistral Small was produced by fine-tuning Mistral Small 3 on traces from Magistral Medium and then applying reinforcement learning; a three-way comparison of fine-tuning only, reinforcement learning only, and both leads the authors to conclude, in contrast to DeepSeek's findings, that reinforcement learning adds to distillation even for a small model. Benchmarks are AIME 2024 and 2025 (pass@1 and maj@64 over 64 runs), MATH-500, GPQA, LiveCodeBench v5 and v6, Aider Polyglot and the text-only subset of Humanity's Last Exam, run at temperature 0.7 with 32k to 40k token limits. On translated versions of AIME 2024 in six languages, Magistral Medium scores 4.3 to 9.9 points below its English result.

Ablations and analysis

The paper reports that training on mathematics alone or code alone transfers to the other domain, examines batch size and advantage normalisation, and finds that text-only reinforcement learning did not degrade, and on some benchmarks improved, the multimodal performance inherited from the base models. Function-calling and instruction-following scores measured on internal benchmarks were maintained. Two approaches are described as unsuccessful: partial rewards for code and entropy targeting. A further experiment applies reinforcement learning to a checkpoint fine-tuned on open-source reasoning traces generated by DeepSeek-R1.

Provenance and confidence

All results are developer-reported; comparison figures for DeepSeek models are quoted from DeepSeek's own paper and were not re-run. The paper contains no safety evaluations. Confidence is medium. The vault copy is the arXiv HTML rendering of version 1.

Reported results

Evaluation results the document states for which there is no separate benchmark page; results on benchmarks that have a page are recorded there and cite this page. Scores and settings are as printed in the document.

Relationships

• supports: Magistral (Mistral)
• related: Mistral Large / Mistral Medium / Mixtral, DeepSeek-R1, Mistral AI