A quantitative one-year audit of Leopold Aschenbrenner's 2024 Situational Awareness essay, written by Nathan Delisle and published on LessWrong in June 2025 (Source: lesswrong.com). Where most prior critiques were qualitative, Delisle structures the audit around Aschenbrenner's order-of-magnitude (OOM)-per-year scaling thesis and checks it against actual 2024–mid-2025 data. His conclusion is that Aschenbrenner's pace of roughly half an order of magnitude of annual progress is "roughly supported by available evidence," with single-cluster scaling and revenue acceleration lagging slightly behind forecast.
Summary of argument
Delisle separates Aschenbrenner's forecast into two layers:
- Drivers — the underlying engines of capability growth: raw compute, algorithmic efficiency, and post-training enhancements.
- Indicators — observable externalities: cluster size, capital investment, chip production, revenue, and electricity consumption.
He summarizes the audit as follows: "Aschenbrenner's predicted pace of roughly half an order-of-magnitude annual progress is roughly supported by available evidence. Single-cluster scaling and revenue acceleration lag slightly behind forecast; drivers and most infrastructure indicators are in-range."
Driver-by-driver audit
| Driver | Verdict (vs Aschenbrenner) |
|---|---|
| Raw compute | Mixed. xAI's Grok 3 exceeded the projected training-compute waypoint by ~0.38 OOM; OpenAI / Anthropic training runs through mid-2025 trailed by 0.2–0.5 OOM. Directionally consistent with ~0.5 OOM/year. |
| Algorithmic efficiency | In-range. Public optimizer / MoE / data-curation improvements consistent with 0.4–0.8 OOM/year, matching Aschenbrenner's ~0.5 OOM/year. |
| Post-training ("unhobbling") | In-range. Evidence supports ~0.56 OOM of effective compute/year from RLHF + tool use + chain-of-thought + agentic scaffolding. Definitionally slippery. |
Indicator-by-indicator audit
| Indicator | Aschenbrenner forecast | 2024–mid-2025 reality |
|---|---|---|
| Capital investment | ~$300B/year hyperscaler capex | ~$320B disclosed for 2025 — on track |
| Accelerator shipments | 12–16M units in 2025 | In-range |
| Chip supply | TSMC advanced-packaging + wafer allocation on track | On track, but HBM has emerged as the binding bottleneck |
| Power demand | Committed AI datacenter capacity at ~3% of US grid | 2–3 GW, matching projections |
| Largest training clusters | ~300 MW single-cluster waypoint | Lags by ~0.25 OOM — Meta ~100k H100s, xAI Colossus on-trend but not at projected scale |
| Revenue | $10B+ annualized early-2025; $20–40B late-2025 | $10B milestone hit early-2025; late-2025 targets unproven at time of writing |
Scope and exclusions
Delisle is explicit that the piece is a quantitative driver/indicator audit. It does not evaluate:
- Superalignment / intelligence-explosion claims
- The "Project" nationalization forecast
- US-China coalition political-economy predictions
- The safety-vs-security rebrand
Delisle notes these are picked up separately by Jamie Harris's EA Forum retrospective, which his piece references.
Provenance and reliability
The retrospective relies on publicly available data through mid-2025; some 2025 numbers were estimates at time of writing. Some quantitative claims, such as the ~0.56 OOM/year post-training contribution, depend on definitions of "effective compute" that are not standardized, which Delisle flags. The driver/indicator decomposition is the methodological feature of the audit, and its source attribution to Aschenbrenner's OOM-per-year framework is preserved throughout.
The audit shares an OOM-driven forecast structure with AI 2027, whose predictions use a similar framework; Delisle's piece predates Daniel Kokotajlo's sibling retrospective Grading AI 2027's 2025 Predictions (AI Futures Project, February 2026). More recent (2025) data points that update Delisle's June 2025 baseline appear in Epoch AI's compute-trends paper Trends in AI Supercomputers — Pilz, Sanders, Rahman, Heim (Epoch AI / Georgetown / GovAI, 2025) and EPRI's power-needs white paper Scaling Intelligence: The Exponential Growth of AI's Power Needs — EPRI White Paper (August 2025). The capability-stitching methodology in A Rosetta Stone for AI Benchmarks — Ho, Denain, Atanasov, Albanie, Shah (Epoch AI / Google DeepMind, 2025) corresponds to the informal approach Delisle's "drivers" audit uses.
Relationships
- supports: Leopold Aschenbrenner (quantitative retrospective on his 2024 essay), Broad Timelines, Scaling Laws (empirical input)
- related: AI 2027 (companion forecast, similar methodology), Grading AI 2027's 2025 Predictions (AI Futures Project, February 2026) (sibling Kokotajlo retrospective), Trends in AI Supercomputers — Pilz, Sanders, Rahman, Heim (Epoch AI / Georgetown / GovAI, 2025) (more recent compute data), Scaling Intelligence: The Exponential Growth of AI's Power Needs — EPRI White Paper (August 2025) (more recent power data), A Rosetta Stone for AI Benchmarks — Ho, Denain, Atanasov, Albanie, Shah (Epoch AI / Google DeepMind, 2025) (methodology successor for capability stitching), Nathan Delisle (planned)
- instance-of: AI Progress Measurement