François Chollet is an AI researcher, formerly at Google Research, who created the Keras deep-learning API and the ARC-AGI benchmark. He co-founded the ARC Prize competition with Mike Knoop and is a public critic of pure scaling as a path to general intelligence.
Background
Chollet created Keras, a widely used deep-learning API, and worked at Google Research. He co-founded the ARC Prize competition with Mike Knoop.
ARC-AGI and the measurement of intelligence
Chollet introduced the ARC-AGI benchmark in "On the Measure of Intelligence" (2019), designed to test general fluid intelligence rather than domain knowledge or scale-dependent pattern-matching. ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems is the 2025 successor.
ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems was released in May 2025 as a response to the saturation of ARC-AGI-1 by test-time-compute systems, notably OpenAI's o3 run in late 2024. Its design is resistant to pattern-matching and remains feasible for untrained humans, operationalizing a criterion of generalization from few examples.
Positions and statements
Chollet is a public critic of pure scaling as a path to general intelligence, arguing that memorization-heavy benchmarks measure the wrong thing and that true generalization requires novel test cases. His framework runs counter to pure-scaling narratives associated with Scaling Laws and The Bitter Lesson on the question of what constitutes progress.
Relationships
- supports: AI Benchmarks and Evaluation
- contradicts: Scaling Laws framing, The Bitter Lesson framing (on what constitutes progress)
- related: AI Software Progress, ARC-AGI-2: A New Challenge for Frontier AI Reasoning Systems