The Kimi K2 Technical Report is the paper accompanying Kimi K2, a 1.04T-parameter, 32B-active Mixture-of-Experts (MoE) model from the Chinese lab Moonshot AI. Authored by the Kimi Team with 199 listed contributors, it was published as arXiv:2507.20534 on July 28, 2025, and revised February 3, 2026. The model weights are released under a CC BY-NC-ND 4.0 license. The report's three principal contributions are the MuonClip optimizer with a QK-Clip weight-rescaling technique, a large-scale agentic data synthesis pipeline, and a joint reinforcement-learning (RL) stage combining a Verifiable Rewards Gym with a self-critique rubric reward. The model is presented as "open agentic intelligence," with performance on agentic and coding benchmarks reported without extended chain-of-thought. (Source: Kimi K2 Technical Report)
Architecture
Kimi K2 is a 61-layer transformer with a hidden dimension of 7,168 and an expert hidden dimension of 2,048. The MoE configuration uses 384 total experts with 8 activated per token (sparsity 48), one shared expert, and no expert grouping, which differs from DeepSeek-V3. The model uses Multi-head Latent Attention (MLA), the same attention family as DeepSeek-V3, but with 64 attention heads — half of DeepSeek-V3's 128. The report states the reduced head count was chosen to cut 128k-context inference FLOPs by 83% at a validation-loss cost of 0.5–1.2%. The model was pre-trained on 15.5T tokens, with context extended from 4k to 32k to 128k via YaRN.
MuonClip optimizer
The report identifies a stability problem: the Muon optimizer improves token efficiency over AdamW, but exploding attention logits appear at scale, and logit soft-caps and QK-Normalization are insufficient for MLA. The proposed mechanism, QK-Clip, rescales the query and key projection weights per head after each update by γ_h = min(1, τ/S_max^h), where S_max^h is the per-head maximum softmax input over the batch and τ is a target threshold. For MLA, only the unshared components (head-specific Q/K and rotary components) are clipped. The report states that without QK-Clip the logits exceed magnitude 1000, while with τ = 100 the logits stabilize at the threshold and then decay, allowing K2 to train on 15.5T tokens with "no observable spikes." The report describes this as a stability mechanism for trillion-parameter MoE pre-training, distinct from the auxiliary-loss-free balancing introduced by DeepSeek-V3.
Agentic training pipeline
The report describes an agentic data synthesis pipeline with three stages plus real-world grounding. A tool repository combines 3,000+ real Model Context Protocol (MCP) tools harvested from GitHub with 20,000+ synthesized domain-specific tools generated via hierarchical evolution. An agent-and-task generation stage produces thousands of synthetic agents (varied system prompts combined with tool combinations) paired with rubric-based tasks. A multi-turn trajectory generation stage uses LLM-generated user personas, a stateful tool simulator with "controlled stochasticity," and LLM-as-judge rubric filtering that retains only successful trajectories. Real execution sandboxes complement simulation for code and SWE tasks, using test-suite pass rates as ground truth. The report presents this as a detailed public account of industrial-scale agentic data synthesis, relevant to Agentic AI.
RL and rubric rewards
The joint RL stage is built around a Verifiable Rewards Gym covering five domains: math/STEM/logic; complex instruction following (a hybrid of code, LLM-judge, and a "hack-check"); faithfulness (a sentence-level hallucination detector); coding and SWE (human-written unit tests mined from pre-training data); and safety (an attack-model / target-model / judge-model prompt-evolution pipeline).
The stage also uses a self-critique rubric reward, in which the K2 critic scores pairs of model responses against a composite rubric mixing core values, prescriptive anti-reward-hacking rubrics, and domain-specific human-annotated rubrics. On-policy RLVR rollouts continuously refine the critic, which the report describes as "a crucial step that distills objective performance signals from RLVR directly into its evaluation model." (Source: Kimi K2 Technical Report) Additional RL-algorithm enhancements include per-sample token budget control with truncation penalties, a PTX auxiliary loss against curated data, and a temperature decay from exploration to exploitation.
License
The weights are released under CC BY-NC-ND 4.0, permitting research and non-commercial use with no derivatives. This is more restrictive than the Apache-2.0 or DeepSeek-style releases common in the Chinese open-weights lineage, and departs from the open-source AI approach other Chinese frontier labs have used.
Benchmarks
The report lists the following benchmark scores as reported by Moonshot AI:
| Benchmark | Score |
|---|---|
| Tau2-Bench | 66.1 |
| ACEBench (En) | 76.5 |
| SWE-Bench Verified | 65.8 |
| LiveCodeBench v6 | 53.7 |
| AIME 2025 | 49.5 |
| GPQA-Diamond | 75.1 |
Reception and context
Kimi K2 is a second well-documented Chinese frontier MoE alongside DeepSeek-V3; the two share MLA and similar sparsity-scaling logic but diverge on attention-head counts, expert grouping, and optimizer choice. The report's emphasis on training-stability and efficiency innovation aligns with the China and the US Are Running Different AI Races analysis of how Chinese labs are competing. Its release also complicates the open-weights picture: the weights are public, but the CC BY-NC-ND license restricts commercial and derivative use, unlike the permissive licenses adopted by several other Chinese frontier labs.