How We Get To AGI: François Chollet’s talk explained

How We Get To AGI is François Chollet’s explanation of why scaling up language models has not produced general intelligence, and which approach he considers promising: systems that tackle an unfamiliar problem by assembling a program suited to it, combining neural network intuition with discrete program search. Delivered on June 16, 2025, at Y Combinator’s AI Startup School, it is the fullest account of his recent position. This page walks through the talk, explains its concepts and cited figures, and reviews what has happened since.

Sources: the video published by Y Combinator and the transcript published by The Singju Post, which was generated automatically and has not been reviewed by the speaker. The transcript contains recognition errors, including names and technical terms; where there is any doubt, the video takes precedence. The ideas below are paraphrased rather than presented as direct quotations. For a shorter overview, read the key ideas from the talk.

The talk at a glance


Title	How We Get To AGI
Speaker	François Chollet, creator of Keras and ARC-AGI, co-founder of ARC Prize and the Ndea research lab
Event	AI Startup School, organized by Y Combinator
Date and location	June 16, 2025, San Francisco
Format	A presentation with slides, delivered in English
Main argument	Intelligence is the efficiency with which a system handles novelty; it requires combining learned intuition with program search
Links	Video · Transcript · 2019 paper

The talk follows three questions introduced at the start: why scaling pretraining has not led to AGI, whether adaptation at test time will get us there, and what comes next. To answer them, Chollet first returns to the definition of intelligence itself.

Chapter 1: twenty years of progress driven by compute

The presentation opens with a curve. According to François Chollet, the cost of computation has fallen by two orders of magnitude per decade since 1940. For a long time, the availability of compute and data constrained what AI could do.

Around 2010, abundant GPU compute and large datasets made deep learning effective, first in computer vision and then in language. Self-supervised training on text subsequently became the dominant approach: the same architecture and method, applied to ever larger models and datasets, produced predictably higher scores. These are the scaling laws presented at the same event, immediately before Chollet’s talk, by Jared Kaplan, whose presentation he references.

Many people inferred that scale would be enough, and that general intelligence would emerge on its own. Chollet says this had been the prevailing belief two years earlier, but that almost no one still defended it in 2025.

Chapter 2: what the benchmarks were actually measuring

In his account, the mistake came from misunderstanding what benchmarks measured. He distinguishes memorized skills, which are static and specific to a task, from fluid intelligence: the ability to make sense of something unfamiliar as it is encountered.

ARC, released in 2019, was intended to make that distinction visible. Chollet estimates that, between 2019 and GPT-4.5, base models increased in scale by roughly a factor of 50,000. Over that period, their ARC-AGI-1 scores rose from 0% to around 10%, while humans score well above 95%. His conclusion is clear: fluid intelligence does not emerge from large-scale pretraining.

Chapter 3: the shift toward test-time adaptation

In 2024, research turned toward test-time adaptation: models that can change their state during inference to adapt to a new task, rather than draw only on fixed knowledge. This includes test-time training, program synthesis, and the synthesis of reasoning chains.

ARC scores began to rise sharply. In December 2024, OpenAI presented o3, in a version fine-tuned on ARC, which reached human-level performance on ARC-AGI-1 for the first time. According to Chollet, every approach performing well on ARC in 2025 used test-time adaptation. Some observers took this as evidence that AGI had already arrived. He disagrees, and the rest of the talk explains why.

Chapter 4: defining intelligence

This is the theoretical core of the talk. It draws on his 2019 paper, On the Measure of Intelligence, discussed in the article François Chollet’s approach to measuring intelligence.

Chollet contrasts two traditions. For Marvin Minsky, AI meant getting machines to perform tasks that humans perform; its current counterpart defines AGI as a model capable of doing most economically useful tasks. For John McCarthy, AI meant dealing with problems the machine had not been prepared for. Chollet sides with McCarthy: intelligence is a process, and skill is its product. Demonstrating skills, even many of them, does not establish intelligence. His analogy contrasts a road network connecting predetermined locations with a construction company that builds the roads still needed.

He formalizes intelligence as a conversion rate between available information, including past experience and knowledge supplied by designers, and the ability to act in new, uncertain future situations. Intelligence is therefore a measure of efficiency. Three related concepts follow:

Concept	Question	Example from the talk
Static skill or fluid intelligence	Is the system applying an existing program or creating one?	A continuum rather than a binary distinction
Scope of application	Does the skill work far beyond what the system has already encountered?	Being able to drive in any city, rather than only within a mapped area
Information efficiency	How much data and practice are needed to acquire the skill?	Greater efficiency means greater intelligence

This has two practical consequences. Human exams are poor measures of AI because they assume the candidate has not already read the answers. Any single measure is also vulnerable to shortcuts: a system can hit the target while missing the underlying objective. Chollet cites the Netflix Prize, whose winning system was too complex to deploy, and chess, where Deep Blue’s victory over Kasparov revealed almost nothing about intelligence. Pursuing skill leads to automation; pursuing fluid intelligence leads toward invention and faster scientific progress, the benefits AGI is supposed to deliver.

Chapter 5: ARC-AGI-1, 2, and 3

ARC-AGI is Chollet’s attempt to measure this kind of intelligence. The ARC-AGI reference page tracks its versions and scores; the table below summarizes how the talk describes them.

Version	Release	What it tests	Key points cited
ARC-AGI-1	2019	Minimal fluid intelligence, using input-output pairs	1,000 unique tasks; requires only the basic knowledge of a 4-year-old child; behaves like a binary test and saturates below human performance
ARC-AGI-2	March 2025	Compositional generalization, designed to challenge reasoning systems	Tested on around 400 people in San Diego; every task solved by at least two people; 0% for base models, 1 to 2% for reasoning with a single chain
ARC-AGI-3	Announced for early 2026	Agency: exploring, learning through interaction, and setting a goal	Unfamiliar games with unfamiliar controls; efficiency-based scoring, with action budgets limited to the human level

Three ideas frame these versions. ARC does not determine whether a system is AGI: it points toward capabilities that remain out of reach. It resisted scaling and, according to Chollet, was the only benchmark to clearly reveal the shift in 2024, while others had already saturated. His final criterion is that AGI has not arrived as long as it remains easy to devise tasks that humans find simple but AI cannot solve, regardless of the compute available.

Chapter 6: the kaleidoscope hypothesis

If the future had nothing in common with the past, no intelligence could make sense of it. But Chollet argues that nothing is ever entirely new: the world contains recurring similarities, from trees to neurons and from electromagnetism to hydrodynamics. He calls this the kaleidoscope hypothesis: apparently endless complexity emerges from recombining a small number of building blocks of meaning.

Intelligence therefore has two components:

Acquiring abstractions: efficiently extracting reusable building blocks from experience, including invariants, structures, and recurring principles.
Recombining them in the moment: selecting and assembling those building blocks into a model suited to the situation at hand.

Efficiency matters in both cases. Needing hundreds of thousands of hours to acquire a simple skill, or enumerating every possible chess move to find the best one, does not qualify as intelligent in this framework. Intelligence requires efficiency in both data and computation. This connects to generalization, a central theme in Chollet’s work.

Chapter 7: what current models are missing

Chollet identifies two gaps that explain why scaling alone has fallen short:

Recombination at test time: models acquired many abstractions during training but remained fixed afterward, able only to apply a model established in advance. Test-time adaptation addresses this gap, which he considers a major step toward AGI.
Efficiency: gradient descent requires three to four orders of magnitude more data than a human needs to learn a simple abstraction. For recombination, the best techniques in 2025 required thousands of dollars in compute to reach human-level performance on ARC-AGI-1, and did not scale to ARC-AGI-2.

The underlying problem is the absence of compositional generalization, precisely what ARC-AGI-2 is designed to measure. Chollet explains this through a distinction between two types of abstraction.

	Type 1: value-centric	Type 2: program-centric
Domain	Continuous	Discrete: programs, and therefore graphs
Comparison	Continuous distance	Exact structural matching, through graph and subgraph isomorphisms
Used for	Perception, pattern recognition, intuition	Reasoning, planning, precision; the kind of work a developer does when refactoring
Handled well by Transformers	Yes	No: sorting a list or adding numbers supplied token by token remains difficult to learn

Both types rely on analogy. In Chollet’s account, all cognition arises from their combination.

Chapter 8: combining intuition and program search

Type 2 requires discrete program search: exploring combinations of operators drawn from a specialized language. According to Chollet, every known system capable of invention relies on discrete search, from designs produced by genetic algorithms in the 1990s to AlphaGo’s move 37 and DeepMind’s AlphaEvolve. The strengths and weaknesses of program synthesis are the reverse of those found in machine learning:

	Machine learning	Program synthesis
Model	A differentiable parametric function: a curve	A graph of symbolic operations
Learning mechanism	Gradient descent	Combinatorial search
Data requirements	High: dense sampling	Low: two or three examples
Main obstacle	Data density	Combinatorial explosion

The proposed solution combines the two, much as humans do when playing chess. Step-by-step calculation, associated with type 2, focuses on the few moves that intuition, associated with type 1, has selected. A neural network’s fast, approximate judgments guide the search and keep combinatorial explosion under control. Chollet compares this to using a map to estimate a distance quickly, instead of exploring every route through a subway network.

Chapter 9: the proposed architecture and Ndea

Chollet then describes the system his lab, Ndea, is working on:

A programmer-like meta-learner writes a suitable program for each new task.
That program combines deep learning modules for type 1 subproblems, such as perception, with algorithmic modules for type 2 subproblems.
It is assembled through discrete search guided by learned intuition about the structure of program space.
The search draws on a global library of abstractions, expanded with each solved problem, much as a developer publishes a library that others can reuse.
The system improves by growing this library and refining its intuition.

The stated goal is AI capable of autonomous invention and discovery, with the aim of accelerating science. The first announced milestone is to solve ARC-AGI using a system that initially knows nothing about ARC.

The figures cited and what they represent

Figure cited in June 2025	Status
Compute costs falling by a factor of 100 per decade since 1940	An order-of-magnitude claim presented by the speaker
Base models increasing in scale by roughly 50,000 times between 2019 and GPT-4.5	The speaker’s estimate
ARC-AGI-1: base models rising from 0% to around 10%; humans scoring above 95%	The speaker’s estimate, to be read alongside results published by ARC Prize
An ARC-fine-tuned version of o3 reaching human-level performance on ARC-AGI-1 in December 2024	A public result announced by OpenAI and ARC Prize
ARC-AGI-2 released in March 2025 and tested on around 400 people	A public fact reported by ARC Prize
Data requirements three to four orders of magnitude higher than for a human	The speaker’s estimate

What has happened since June 2025

Confirmed: ARC-AGI-3 was released in March 2026. Its technical report states that humans could solve every environment at that point, while leading AI systems scored below 1%, in line with Chollet’s expectations.

Qualified: by October 2026, verified ARC Prize results show very high ARC-AGI-3 scores, but those scores depend heavily on the agent harness. Gemini 4 Argon scores 97.5% with the harness supplied by its developer and 49.8% with the standard ARC Prize harness; GPT-6.1 Sol shows a gap of a similar magnitude. Our benchmark comparison tool tracks these figures with each update.

Still open: efficiency was central to the talk. How much of a score comes from the model, how much comes from its tooling, and at what cost? These are exactly the issues the talk raised. Chollet’s final criterion, the increasing difficulty of inventing tasks that are easy for humans but impossible for AI, cannot be captured by a single score.

What remains disputed

These arguments are not universally accepted, and the talk itself identifies the main points of disagreement:

The role of scale: the scaling laws presented at the same event remain central to several labs’ strategies, supplemented by reinforcement learning and inference-time compute. Chollet does not dispute their effectiveness for automation; he disputes whether they are sufficient for fluid intelligence.
The definition of AGI: the economic definition, based on performing most useful tasks, and Chollet’s definition, based on adapting efficiently to novelty, lead to different measurements and timelines. AGI forecasts diverge partly for this reason.
Measurement: ARC-AGI is itself a benchmark and is subject to the limitations described in the talk. This is why it has successive versions and a private evaluation dataset. See measuring progress toward AGI.

Glossary for the talk

Fluid intelligence: the ability to solve a new problem, as distinct from applying acquired skills.
Test-time adaptation: changing a model’s state during inference to adjust to a task.
Abstraction: a reusable building block extracted from experience by removing details that do not matter.
Compositional generalization: the ability to combine familiar building blocks to handle an unfamiliar case.
Discrete program search: exploring combinations of operators to find a program that solves the task.
Meta-learner: a system that learns to construct, for each task, the model that will solve it.

Frequently asked questions

When and where did François Chollet give How We Get To AGI?

He gave the talk on June 16, 2025, at AI Startup School, an event organized by Y Combinator in San Francisco. Y Combinator published the video on its YouTube channel in early July 2025.

Do you need a technical background to follow the talk?

The main ideas are accessible without technical training: the definition of intelligence, the role of ARC-AGI, and the kaleidoscope hypothesis. The second half, covering type 1 and type 2 abstractions and program search, assumes a basic understanding of neural networks and programs. The tables on this page explain the essentials.

How does this page differ from the blog article about the talk?

The blog article offers a brief overview of the main arguments. This page is a reference guide, with a chapter-by-chapter walkthrough, a glossary, a table explaining the status of the figures cited, and a review of what has or has not held up since June 2025.

Have the figures in the talk been verified?

These are figures presented by the speaker in June 2025 and should be read in that context. Some are estimates, including the 50,000-fold increase in model scale and the data gap between models and humans. Others are publicly verifiable facts, such as the release date of ARC-AGI-2 and the o3 result from December 2024. The table on this page distinguishes between them.

Has ARC-AGI-3 been solved as of October 2026?

Not in the sense discussed in the talk. The best models exceed 95% with an agent harness supplied by their developer, but remain around 50% with the standard ARC Prize harness, while the competition scored on the private dataset continues. Efficiency, a central concern of the talk, remains an open question.

How We Get To AGI: François Chollet’s talk, chapter by chapter