Qwen3

Qwen3 is a family of large language models released in May 2025 by Alibaba's Qwen team. It comprises eight models spanning 0.6B to 235B total parameters, in both dense and mixture-of-experts (MoE) configurations, all under the Apache 2.0 license. Its defining design feature is a unified framework in which a single model operates in either a "thinking" or "non-thinking" inference mode, with a user-controlled thinking budget set at request time (Source: Qwen3 Technical Report).

Model family

Eight models were released simultaneously (Source: Qwen3 Technical Report):

Context length is 32K native, extensible to 128K (Source: Qwen3 Technical Report).

The line was subsequently extended. In April 2026 Alibaba shipped Qwen3.6-27B as an open-weights model (Source: qwenlm.github.io).

Unified thinking / non-thinking with thinking budget

Qwen3 packages two inference modes in a single model (Source: Qwen3 Technical Report):

• Thinking mode — extended chain-of-thought reasoning for complex, multi-step tasks.
• Non-thinking mode — fast, direct responses for everyday queries.
• Thinking budget — users specify a thinking-token threshold at inference, letting applications adjust reasoning depth per request.

The technical report describes this design as eliminating the deployment complexity of running separate chat-optimized models (such as GPT-4o-style models) and dedicated reasoning models (such as DeepSeek-R1-style models). It contrasts with OpenAI's o-series and DeepSeek's R1, which are distinct models from their non-reasoning siblings. In effect, the "reasoning model vs. base model" distinction used by some leaderboards (see GPQA: A Graduate-Level Google-Proof Q&A Benchmark and AI Benchmarks and Evaluation) becomes a within-model setting rather than a choice between models, and the budget control suits agentic systems that interleave cheap fast steps with occasional deeper reasoning.

The Qwen team subsequently reversed this design choice. Announcing Qwen3-235B-A22B-Instruct-2507 (with an FP8 variant) in July 2025, the team said that after community feedback it would stop using hybrid thinking mode and instead train Instruct and Thinking models separately to obtain the best quality from each (Source: x.com). Later Qwen3-family releases accordingly ship as separate instruct and thinking variants rather than a single dual-mode model (Source: x.com).

Training and architecture

The models were trained on 36 trillion tokens drawn from general knowledge, code, STEM, reasoning, books, multilingual text, and synthetic data (Source: Qwen3 Technical Report). Synthetic data was produced by specialist Qwen2.5 variants: Qwen2.5-VL extracts text from PDFs, while Qwen2.5-Math and Qwen2.5-Coder generate domain-specific synthetic data. This in-family generation loop is similar in structure to the R1→V3 reasoning distillation described in DeepSeek-V3 Technical Report.

Pre-training proceeded in three stages (Source: Qwen3 Technical Report):

1. General (S1): more than 30T tokens at 4,096 sequence length.
2. Reasoning (S2): roughly 5T higher-quality tokens emphasizing STEM, code, and reasoning.
3. Long context: hundreds of billions of tokens at 32,768 sequence length, with 75% of text in the 16K–32K range.

Post-training proceeded in four stages (Source: Qwen3 Technical Report):

1. Long-CoT cold-start supervised fine-tuning.
2. Reasoning RL with GRPO on 3,995 query-verifier pairs. GRPO is the same algorithm used at DeepSeek (see DeepSeek-V3).
3. Thinking Mode Fusion SFT, combining thinking and non-thinking examples so a single model handles mode switching.
4. General-domain RL.

Smaller Qwen3 models are produced by strong-to-weak Distillation from larger ones, requiring only roughly 1/10 of the GPU hours of the full four-stage post-training pipeline (Source: Qwen3 Technical Report). This in-lab distillation parallels the pattern documented at DeepSeek-V3.

Benchmark performance

The figures below are reported by the technical report (Source: Qwen3 Technical Report) and have not been independently replicated here, so they are treated as medium-confidence. See GPQA: A Graduate-Level Google-Proof Q&A Benchmark and AI Benchmarks and Evaluation for evaluation context and caveats.

Qwen3-235B-A22B-Base against peers:

Qwen3-32B-Base reports MMLU-Pro 65.54 (versus Qwen2.5-72B's 58.07), MATH 61.62, and EvalPlus 72.05. On generational efficiency, Qwen3-8B is reported as comparable to Qwen2.5-14B across 15 benchmarks, and Qwen3-4B is reported to exceed Qwen2.5-7B on most tasks.

License and release

All eight models, including the 235B flagship, were released under Apache 2.0 (Source: Qwen3 Technical Report). This is a more permissive license than Meta's Llama community license, placing Qwen3 among the most openly licensed frontier-scale models reported to date and bearing directly on the Open-Source AI / Open-Weight Models debate. A flagship-scale Apache 2.0 release runs counter to the framing that Alibaba was moving toward closed releases, and extends the pattern of Chinese labs shipping open weights at large scale established by DeepSeek-V3 and DeepSeek-R1 (see US-China AI Competition). Combined with 119-language coverage, the permissive licensing also makes Qwen3 a candidate substrate for non-US, non-China national deployments (see AI Sovereignty).

Deployment

In April 2026 Alibaba announced at the Beijing Auto Show that its Qwen AI model would be integrated into car models from BYD, Volkswagen, and other manufacturers, allowing drivers to order food and book hotel rooms by voice command (Source: cnbc.com).

Relationships

• instance-of: General-Purpose AI (GPAI)
• related: DeepSeek-R1, DeepSeek-V3, Open-Source AI / Open-Weight Models, Distillation, China and the US Are Running Different AI Races, AI Sovereignty
• depends-on: Scaling Laws, Distillation

Sources

Primary source: Qwen3 Technical Report. Related technical context: DeepSeek-V3 Technical Report, DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning, (Source: epochai.substack.com), China and the US Are Running Different AI Races, Open-Source AI / Open-Weight Models.