"The Least Understood Driver of AI Progress" is an analysis by Anson Ho, published by Epoch AI in its Gradient Updates series, examining AI "software progress" — the rate at which the training compute required to reach a given capability level declines over time. The piece argues that this factor is among the most important and least understood in AI development trajectories, estimates its rate, identifies its underappreciated drivers, and assesses what those findings imply for the possibility of a software intelligence explosion.
Summary of argument
Ho defines software progress as the decline over time in the training compute needed to reach a fixed capability level, and treats it as a central but poorly measured input to forecasts of AI development. The analysis combines an estimate of the rate of software progress with an account of what mechanisms drive it, and uses both to revisit standard arguments about whether automated AI research could produce an intelligence explosion.
Rate of software progress
Ho's best estimate is roughly 10× per year — the training compute needed to reach the same capability declines by about an order of magnitude annually — with an 80% confidence interval of 2× to 50× per year. The piece characterizes the surrounding uncertainty as "almost comically wide," noting that estimates across the literature range from 1.1× to 300× per year. It also argues that most existing estimates fail to capture post-training improvements such as reasoning methods and RLHF, which Ho describes as a now-dominant paradigm.
Drivers of software progress
Ho identifies two factors he regards as underappreciated in explaining measured rates of software progress.
The first is improvements in data quality, as distinct from algorithmic advances. The shift from uncurated web data to synthetic or processed data, together with distillation that allows smaller models to reach the capabilities of larger ones using a fraction of the compute, may account for the bulk of measured efficiency gains.
The second is scale-dependent innovations. A small number of key innovations — the transitions from LSTM to Transformer and from Kaplan to Chinchilla scaling — produce larger efficiency gains at higher compute scales. At 10^21 FLOP, Ho cites a modern Transformer as 6.3× more efficient than an LSTM; at 10^25 FLOP, the gain is 26×. On this view, most measured software progress may come from scaling up just two or three such innovations rather than from discovering many new algorithms.
Implications for the software intelligence explosion
Ho presents the rate estimate as cutting in both directions on the likelihood of a software intelligence explosion. It makes an explosion more likely in that prior analyses used more conservative estimates of around 3× per year; correcting to roughly 10× per year increases estimated "returns to AI software R&D," pushing above the threshold associated with an intelligence explosion. It makes an explosion less likely in that existing models ignore the dependence of software progress on scaling training compute. If most gains come from scale-dependent innovations, automated AI researchers cannot make fast progress without also scaling compute, producing what Ho calls a "compute bottleneck."
On net, Ho leans toward viewing the compute bottleneck as real and the software intelligence explosion as less likely than previously estimated, while characterizing the evidence as uncertain.
The analysis applies this framing to three questions. On DeepSeek catching up to o1, Ho argues that if software improves about 10× per year, a few months of progress combined with distillation makes the catch-up less surprising. On when AGI will arrive, he argues that software progress matters enormously and that Cotra's bioanchors model was roughly a decade too long because it underestimated software progress. On whether automated AI research will explode, he frames the outcome as depending on returns to R&D (possibly high) against compute bottlenecks (possibly binding); Ho states he is skeptical of "5 orders of magnitude of software progress in a year" but assigns it roughly a 15% probability.
Relationships
- supports: The Bitter Lesson — provides empirical evidence for, and complications of, Sutton's thesis that general methods leveraging computation tend to win.
- related: Fast-Follow Problem — a ~10×/year rate of software progress bears on how easily a leading model can be matched.
- supports: Compute Governance — the compute-bottleneck finding bears on the case that controlling compute may also constrain the rate of software progress.
- related: AI 2027 — the intelligence-explosion analysis engages directly with the AI 2027 scenario.
- contradicts: AI as Normal Technology — fast software progress presses on the "normal technology" thesis, while the scale-dependence finding offers a counterpoint that progress is not free of expensive compute.