Three year-in-review blog posts by Simon Willison, each published December 31 of 2023, 2024, and 2025 on simonwillison.net. Written from the perspective of a developer using language models daily rather than that of a lab executive or academic, they track the evolution of LLMs from GPT-4 exclusivity through price collapse and commoditization, the rise of reasoning models and coding agents, and the normalization of AI-assisted development. The three posts are treated here as one continuous narrative.
2023: "Stuff We Figured Out About AI"
The 2023 review argues that LLMs are easier to build than expected — a few hundred lines of Python suffice for basic training, with training costs dropping from millions to tens of thousands of dollars — and that they can be run on personal devices, which Willison describes as the start of local-model culture. As of late 2023, he notes that no organization other than OpenAI had produced a GPT-4-class model. He characterizes LLMs as "really smart, and also really, really dumb," identifying gullibility as the biggest unsolved problem, suggests that code may be their best application, and describes the surrounding ethics as "diabolically complex."
2024: "Things We Learned About LLMs"
The 2024 review states that the GPT-4 barrier was comprehensively broken: 18 organizations had models beating GPT-4, and GPT-4 itself fell to roughly 70th place on Chatbot Arena. GPT-4-class models could by then run on a laptop, with Willison citing a 64GB MacBook Pro running Llama 3.3 70B. He reports that LLM prices crashed due to competition and efficiency gains, that multimodal vision became common while audio and video were emerging, and that "agents" still had not really happened yet. The post describes the rise of inference-scaling "reasoning" models such as o1 and o1-mini, and raises the question of DeepSeek: "Was the best currently available LLM trained in China for less than $6m?" Willison writes that the environmental impact "got better" through efficiency and "got much, much worse" through scale, labels 2024 "the year of slop" for the AI-generated content flooding the internet, and observes that knowledge about how to use LLMs is "incredibly unevenly distributed."
2025: "The Year in LLMs"
The 2025 review frames the year around reasoning and agents. Willison writes that every notable lab released reasoning models, with the real unlock coming from driving tools for multi-step planning, and that coding agents such as Claude Code became genuinely useful. He describes growing casualness about AI use as "the year of YOLO and normalization of deviance," notes the emergence of a $200/month premium subscription tier, and reports top-ranked Chinese open-weight models from DeepSeek and others. He characterizes 2025 as the year that OpenAI lost their lead as other labs caught up, and as "the year of Gemini," with Google's model family becoming competitive. The post discusses "vibe coding" — programming guided by intuition rather than deep understanding — and the spread of MCP (Model Context Protocol) for standardizing tool use. It introduces "the lethal trifecta," in which prompt injection combined with tool access and insufficient permissions produces a security risk, and observes that data centers became extremely unpopular as community resistance grew.
Cross-cutting themes
Across the three posts, Willison traces an arc of democratization: in 2023 only OpenAI had GPT-4; by 2024, 18 organizations had beaten it and GPT-4-class models ran on a laptop; by 2025, Chinese labs were producing top-ranked open-weight models. This bears on the Fast-Follow Problem, suggesting that catching up has been faster than expected.
Each year, Willison notes a gap between what models can do on benchmarks and what they actually deliver in practice. This connects to Narayanan & Kapoor's "AI as normal technology" argument about construct-validity problems in benchmarks.
The "slop" theme — AI-generated low-quality content flooding the internet — recurs across 2024 and 2025 as an example of AI Diffusion producing negative externalities that existing governance has not addressed.
Provenance
Raw Sources/Stuff we figured out about AI in 2023.mdRaw Sources/Things we learned about LLMs in 2024.mdRaw Sources/2025 The year in LLMs.md