Kimi K2 Technical Report

The Kimi K2 Technical Report is the paper accompanying Kimi K2, a 1.04T-parameter, 32B-active Mixture-of-Experts (MoE) model from the Chinese lab Moonshot AI. Authored by the Kimi Team with 199 listed contributors, it was published as arXiv:2507.20534 on July 28, 2025, and revised February 3, 2026. The model weights are released under a CC BY-NC-ND 4.0 license. The report's three principal contributions are the MuonClip optimizer with a QK-Clip weight-rescaling technique, a large-scale agentic data synthesis pipeline, and a joint reinforcement-learning (RL) stage combining a Verifiable Rewards Gym with a self-critique rubric reward. The model is presented as "open agentic intelligence," with performance on agentic and coding benchmarks reported without extended chain-of-thought. (Source: Kimi K2 Technical Report)

Architecture

Kimi K2 is a 61-layer transformer with a hidden dimension of 7,168 and an expert hidden dimension of 2,048. The MoE configuration uses 384 total experts with 8 activated per token (sparsity 48), one shared expert, and no expert grouping, which differs from DeepSeek-V3. The model uses Multi-head Latent Attention (MLA), the same attention family as DeepSeek-V3, but with 64 attention heads — half of DeepSeek-V3's 128. The report states the reduced head count was chosen to cut 128k-context inference FLOPs by 83% at a validation-loss cost of 0.5–1.2%. The model was pre-trained on 15.5T tokens, with context extended from 4k to 32k to 128k via YaRN.

MuonClip optimizer

The report identifies a stability problem: the Muon optimizer improves token efficiency over AdamW, but exploding attention logits appear at scale, and logit soft-caps and QK-Normalization are insufficient for MLA. The proposed mechanism, QK-Clip, rescales the query and key projection weights per head after each update by γ_h = min(1, τ/S_max^h), where S_max^h is the per-head maximum softmax input over the batch and τ is a target threshold. For MLA, only the unshared components (head-specific Q/K and rotary components) are clipped. The report states that without QK-Clip the logits exceed magnitude 1000, while with τ = 100 the logits stabilize at the threshold and then decay, allowing K2 to train on 15.5T tokens with "no observable spikes." The report describes this as a stability mechanism for trillion-parameter MoE pre-training, distinct from the auxiliary-loss-free balancing introduced by DeepSeek-V3.

Agentic training pipeline

The report describes an agentic data synthesis pipeline with three stages plus real-world grounding. A tool repository combines 3,000+ real Model Context Protocol (MCP) tools harvested from GitHub with 20,000+ synthesized domain-specific tools generated via hierarchical evolution. An agent-and-task generation stage produces thousands of synthetic agents (varied system prompts combined with tool combinations) paired with rubric-based tasks. A multi-turn trajectory generation stage uses LLM-generated user personas, a stateful tool simulator with "controlled stochasticity," and LLM-as-judge rubric filtering that retains only successful trajectories. Real execution sandboxes complement simulation for code and SWE tasks, using test-suite pass rates as ground truth. The report presents this as a detailed public account of industrial-scale agentic data synthesis, relevant to Agentic AI.

RL and rubric rewards

The joint RL stage is built around a Verifiable Rewards Gym covering five domains: math/STEM/logic; complex instruction following (a hybrid of code, LLM-judge, and a "hack-check"); faithfulness (a sentence-level hallucination detector); coding and SWE (human-written unit tests mined from pre-training data); and safety (an attack-model / target-model / judge-model prompt-evolution pipeline).

The stage also uses a self-critique rubric reward, in which the K2 critic scores pairs of model responses against a composite rubric mixing core values, prescriptive anti-reward-hacking rubrics, and domain-specific human-annotated rubrics. On-policy RLVR rollouts continuously refine the critic, which the report describes as "a crucial step that distills objective performance signals from RLVR directly into its evaluation model." (Source: Kimi K2 Technical Report) Additional RL-algorithm enhancements include per-sample token budget control with truncation penalties, a PTX auxiliary loss against curated data, and a temperature decay from exploration to exploitation.

License

The weights are released under CC BY-NC-ND 4.0, permitting research and non-commercial use with no derivatives. This is more restrictive than the Apache-2.0 or DeepSeek-style releases common in the Chinese open-weights lineage, and departs from the open-source AI approach other Chinese frontier labs have used.

Benchmarks

The report lists the following benchmark scores as reported by Moonshot AI:

Reception and context

Kimi K2 is a second well-documented Chinese frontier MoE alongside DeepSeek-V3; the two share MLA and similar sparsity-scaling logic but diverge on attention-head counts, expert grouping, and optimizer choice. The report's emphasis on training-stability and efficiency innovation aligns with the China and the US Are Running Different AI Races analysis of how Chinese labs are competing. Its release also complicates the open-weights picture: the weights are public, but the CC BY-NC-ND license restricts commercial and derivative use, unlike the permissive licenses adopted by several other Chinese frontier labs.

Reported results

Evaluation results the document states for which there is no separate benchmark page; results on benchmarks that have a page are recorded there and cite this page. Scores and settings are as printed in the document.

Discrepancies noted

Table 3 gives Qwen3-235B-A22B's PaperBench Code-Dev accuracy as 13.2 where the text of Appendix C gives 8.2%.

Relationships

• related: DeepSeek-V3 Technical Report, Qwen3 Technical Report, China and the US Are Running Different AI Races, Open-Source AI / Open-Weight Models, Agentic AI