François Chollet is an AI researcher, formerly at Google Research, who created the Keras deep-learning API and the ARC-AGI benchmark. He co-founded the ARC Prize competition with Mike Knoop and is a public critic of pure scaling as a path to general intelligence.
Background
Chollet created Keras, a widely used deep-learning API, and worked at Google Research. He co-founded the ARC Prize competition with Mike Knoop.
ARC-AGI and the measurement of intelligence
Chollet introduced the ARC-AGI benchmark in "On the Measure of Intelligence" (2019), designed to test general fluid intelligence rather than domain knowledge or scale-dependent pattern-matching. ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems is the 2025 successor.
ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems was released in May 2025 as a response to the saturation of ARC-AGI-1 by test-time-compute systems, notably OpenAI's o3 run in late 2024. Its design is resistant to pattern-matching and remains feasible for untrained humans, operationalizing a criterion of generalization from few examples.
The third generation, ARC-AGI-3, was posted to arXiv in March 2026 and launched the same month, with Chollet credited as benchmark designer, co-founder and board member of the ARC Prize Foundation, and listed as the submitting author. It moves from static grid puzzles to interactive turn-based environments and scores agents on action efficiency relative to a human first-run baseline rather than on completion. The report states the foundation's belief that ARC-AGI-1 and ARC-AGI-2 data has entered frontier training corpora, and sets out its position that the official leaderboard will report neither public-set scores nor harness-assisted results (ARC-AGI-3: A New Challenge for Frontier Agentic Intelligence).
Positions and statements
Chollet is a public critic of pure scaling as a path to general intelligence, arguing that memorization-heavy benchmarks measure the wrong thing and that true generalization requires novel test cases. His framework runs counter to pure-scaling narratives associated with Scaling Laws and The Bitter Lesson on the question of what constitutes progress.
Relationships
- supports: AI Benchmarks and Evaluation
- contradicts: Scaling Laws framing, The Bitter Lesson framing (on what constitutes progress)
- related: AI Software Progress, ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems, ARC-AGI-3: A New Challenge for Frontier Agentic Intelligence
- depends-on: ARC-AGI — the benchmark series he introduced and the definition of intelligence behind it