Scaling laws are empirical relationships showing that AI model performance improves predictably as a power law of training compute, dataset size, and model parameters. They are the basis for the scaling hypothesis, the proposition that making neural networks larger with more data and compute yields better performance.
Background and key results
Two results anchor the literature:
- Kaplan scaling laws (OpenAI, 2020): the first systematic demonstration that language model loss decreases as a power law of compute, data, and parameters. They favored larger models trained on less data.
- Chinchilla scaling laws (Google DeepMind, 2022): showed the Kaplan laws used suboptimal data-to-parameter ratios. Chinchilla rebalancing, using more data relative to parameters, produced the same performance with less compute. This insight is among the largest measured sources of AI software progress.
Relation to other concepts
Scaling laws connect to several adjacent concepts:
- The Bitter Lesson — Sutton's 2019 essay is the philosophical statement of what scaling laws demonstrate empirically: general methods leveraging computation outperform human-knowledge approaches.
- AI Software Progress — much of the measured ~10×/year efficiency gain comes from scale-dependent innovations that produce larger gains at higher compute scales (Source: The Least Understood Driver of AI Progress).
- Compute Governance — if performance follows scaling laws, then controlling compute controls capability.
- Compressed 21st Century — Amodei's predictions rest on the assumption that scaling laws continue to hold.
- The Scaling Era — oral history of how the scaling hypothesis moved from minority view to dominant paradigm.
Forecasting model performance
Stanford computer scientists Sanmi Koyejo and Sang Truong introduced a scaling-law framework borrowed from standardized-testing psychometrics, called Item Response Scaling Laws, in work published by Stanford HAI on May 21, 2026. The framework cuts the compute needed to predict large-model performance by as much as 99%. The mechanism is item-response theory, used in standardized testing, which lets researchers estimate a model's latent ability from a smaller, more carefully selected set of evaluation items rather than running the full benchmark at scale. The effect applies to capability forecasting rather than capability production: even where training compute is constrained, evaluation compute, which is itself a meaningful fraction of frontier-lab compute budgets, can be compressed.
If the technique holds, the capability-evaluation overhead in a lab's RSP would drop, making frequent, cheap pre-deployment evaluations more tractable, and it could alter the compute-cost framing of the pre-release-vetting debate if it generalizes from psychometric benchmarks to dangerous-capability evaluations. (Source: hai.stanford.edu)
Debates and positions
Whether scaling laws will continue is contested, with several positions:
- Pro-scaling: Amodei argues scaling laws have a decade-long track record and show no signs of stopping. AI 2027 assumes continued scaling to project superhuman AI by the end of the decade. In remarks at Stanford circulated June 22, 2026, Sam Altman argued that years of skepticism toward scaling large language models slowed the field, citing a recent OpenAI model that he said disproved a mathematical conjecture; Amodei agreed that raw scaling remains the primary engine of progress (Source: aibreakfast.beehiiv.com).
- Anti-scaling: brain-inspired architectures suggest the scaling paradigm may face physical limits. The normal technology framework argues benchmarks systematically overestimate real-world impact.
- Nuanced: Epoch AI's analysis argues that measured scaling progress is partially an artifact of data-quality improvements and scale-dependent architectural innovations, not purely a function of compute increase.
- Physical-input (bottleneck) view: Epoch's "Can AI Scaling Continue Through 2030?" argues that continuing at ~4×/year is physically achievable but strains every input simultaneously. Power is the tightest constraint, with the largest runs projected at 4–16 GW by 2030 per Epoch AI — How Much Power Will Frontier AI Training Demand in 2030? and 2–10 GW per RAND — AI's Power Requirements Under Exponential Growth (2025); CoWoS-L packaging and HBM are the next; high-quality data may exhaust 2026–2028; latency is not binding before 2030. By this account, failure of any one input slows rather than stops scaling, but shifts the competitive landscape.
Hardware and power as a bottleneck
The physical-input analysis adds a bottleneck axis to the scaling debate that earlier framings treated architecturally:
- Power trajectory: training-run site power has grown ~2.2×/year, faster than hardware-efficiency gains of ~1.3×/year FLOP/watt. Unless this gap closes, per-run power grows ~1.7×/year in real terms.
- Grid buildout: sustained scaling through 2030 requires ~3× historical rates of US capacity additions.
- Supply chain: advanced packaging (TSMC CoWoS-L) and HBM3e/HBM4 are the tightest chip-side links.
- Policy coupling: per Compute Governance, power has become a first-class governance lever partly because scaling laws make it a binding input.
If scaling laws hold and physical inputs can be marshalled, the compressed 21st century scenarios become more plausible; if either breaks, the scaling-era dynamics described in The Scaling Era: An Oral History of AI, 2019–2025 — Chapter 1: Scaling enter a new regime.
See also
- related: Mixture of Experts (MoE) — sparse MoE architecture decouples parameter count from active compute, creating a new axis for scaling-law optimization.
- related: Inference Economics — inference-time scaling is the complement to training-time scaling; deflation dynamics operate on both.