Qwen3 Technical Report

The Qwen3 Technical Report is the paper describing Qwen3, Alibaba's May-2025 open-weight model family. It was published on arXiv (2505.09388) on May 14, 2025 by the Qwen Team, Alibaba Cloud (An Yang et al.; roughly 60 authors). The report covers eight models spanning dense (0.6B–32B) and Mixture-of-Experts (30B-A3B, 235B-A22B) architectures, all released under the Apache 2.0 license.

Summary of findings

The report's main design contribution is a unified thinking / non-thinking framework: a single model that can switch between extended chain-of-thought reasoning and rapid direct response, with a user-controllable thinking budget that trades inference latency for reasoning depth. The report describes this as removing the need for separate chat-optimized and reasoning models in a deployment stack. (Source: Qwen3 Technical Report)

The report also expands multilingual coverage from 29 languages (Qwen2.5) to 119 languages and dialects, and releases all weights under Apache 2.0. (Source: Qwen3 Technical Report)

Key claims

• Unified thinking/non-thinking framework with thinking budget. A single model handles both reasoning and non-reasoning modes; users specify a thinking token threshold at inference to control the cost/quality trade-off. (high) (Source: Qwen3 Technical Report)
• Eight-model family, 0.6B–235B, dense and MoE. Dense: 0.6B, 1.7B, 4B, 8B, 14B, 32B. MoE: Qwen3-30B-A3B (3B active) and the flagship Qwen3-235B-A22B (22B active), both with 128 experts / 8 activated. (high) (Source: Qwen3 Technical Report)
• 36 trillion training tokens across a three-stage pre-training curriculum (general → reasoning → long context), followed by a four-stage post-training pipeline (Long-CoT SFT → reasoning RL via GRPO → Thinking Mode Fusion SFT → general RL). (high) (Source: Qwen3 Technical Report)
• Strong-to-weak Distillation for smaller models uses roughly 1/10 of the GPU hours of the full four-stage pipeline. (high) (Source: Qwen3 Technical Report)
• 119 languages and dialects supported, up from 29 in Qwen2.5. (high) (Source: Qwen3 Technical Report)
• Apache 2.0 release for all models, including the 235B flagship. (high) (Source: Qwen3 Technical Report)
• Benchmark claims: Qwen3-235B-A22B-Base reports MMLU 87.81, EvalPlus 77.60, MATH 71.84, GPQA 47.47, which the report states match or exceed DeepSeek-V3 (MMLU 87.19, EvalPlus 63.75) on reported bases. Qwen3-8B is claimed comparable to Qwen2.5-14B across 15 benchmarks. (medium — internally reported; no independent replication recorded) (Source: Qwen3 Technical Report)

Architecture

• Dense models for sizes ≤32B; MoE for the two largest.
• MoE configuration: 128 total experts, 8 activated per token (consistent across 30B-A3B and 235B-A22B).
• Context lengths: 32K natively, extensible to 128K.

Training pipeline

Pre-training proceeds in three stages:

1. General Stage (S1): more than 30T tokens at 4,096 sequence length.
2. Reasoning Stage (S2): roughly 5T higher-quality tokens (STEM, code, reasoning).
3. Long Context Stage: hundreds of billions of tokens at 32,768 sequence length, with 75% of text in the 16K–32K range.

Post-training proceeds in four stages: Long-CoT cold-start SFT → GRPO RL on 3,995 query-verifier pairs → Thinking Mode Fusion SFT (mixed thinking and non-thinking examples) → general-domain RL.

For synthetic data, Qwen2.5-VL extracts text from PDFs, and Qwen2.5-Math and Qwen2.5-Coder produce synthetic domain data, forming a self-improvement loop within the Qwen family.

Licensing

The report releases all eight models under Apache 2.0. This contrasts with Meta's Llama community license (not OSI-approved) and with closed-weight releases from Anthropic, OpenAI, and Google. Qwen3 is described as among the most permissive frontier-scale releases reported to date.

Benchmarks

The raw source file contains the full tables. Selected reported figures:

All numbers are per the technical report. Internal benchmark figures from the paper are treated as medium-confidence pending independent replication; architectural, training-pipeline, and licensing claims are high-confidence, being directly stated in the paper and consistent with the released weights.

Context and reception

The report places Qwen3 alongside other open-weight releases from Chinese labs. The Apache 2.0 release of the 235B flagship extends the pattern set by DeepSeek-V3 and DeepSeek-R1, in which Chinese labs ship frontier-competitive open-weight models under permissive licenses; the 235B flagship under Apache 2.0 is a more permissive stance than Meta's Llama community license. This runs against the framing that China races to close while the US races to open (US-China AI competition).

The Apache 2.0 release of a 235B MoE model also bears on open-source AI: the open-source page records the position, drawn from secondary sources, that "Alibaba has recently tilted back toward closed releases," which Qwen3 at least partially contradicts, as Alibaba continued to ship frontier-scale open weights as of May 2025 (Source: epochai.substack.com).

The strong-to-weak distillation reported inside the Qwen family (roughly 10× GPU-hour savings for smaller models) parallels the DeepSeek-V3 pattern of producing smaller models and the R1-to-V3 reasoning distillation described in DeepSeek-V3 Technical Report, bearing on distillation as a technique used within a single lab as well as across labs.

The unified thinking/non-thinking design addresses the reasoning-model split (o1-style versus chat-style) at the deployment level, and complicates benchmarks framed around distinct "reasoning" and "non-reasoning" model classes, relating to scaling laws and reasoning models. Its 119-language support relates to AI sovereignty arguments, since non-US, non-China national deployments can run Qwen3 under Apache 2.0 without an API dependency.

Relationships

• supports: Qwen3, Alibaba / Qwen Team, Open-Source AI / Open-Weight Models, Distillation, China and the US Are Running Different AI Races
• related: DeepSeek-V3 Technical Report, China and the US Are Running Different AI Races, Open-Source AI / Open-Weight Models, Scaling Laws
• depends-on: Distillation, General-Purpose AI (GPAI)