Deep learning limits: understanding AI models

Deep learning can deliver strong performance while remaining fragile when rules change, skills must be combined, or an answer needs independent verification. François Chollet examines this difficulty by distinguishing acquired skills from efficiency in learning a new task; other researchers study statistical shortcuts, causality, and architectures capable of planning. This research documents difficulties and explores possible solutions, without proving that every neural network faces a permanent barrier.

What kinds of limits are we discussing?

Deep learning learns representations from data by adjusting the parameters of neural networks. It encompasses more than conversational assistants or text prediction: modalities, objectives, and architectures vary. A criticism of a language model therefore does not automatically apply to the entire field.

Several levels need to be distinguished. An architecture may be able to represent a solution, yet training may fail to discover it. A solution may be learned but require too much data or computation. Finally, a system may succeed frequently without providing the reliability its intended use requires.

The word limit can therefore refer to different findings:

an error observed on a specific task under a specific protocol;
a recurring weakness across several conditions;
a cost that makes a method impractical;
an impossibility proven under explicit assumptions.

Moving from the first claim to the last requires justification. An assistant’s failure to solve a problem does not prove a mathematical impossibility. Conversely, showing that a network could theoretically compute a function does not establish that realistic training will teach it to do so.

This page focuses mainly on limits to generalization and reliability. It complements the AGI and research overview, which separates measurement questions from forecasts. The practical examples below are hypothetical situations intended to clarify evaluation criteria, rather than reports of experimental results.

François Chollet: distinguishing skill from intelligence

In On the Measure of Intelligence, published in November 2019, François Chollet proposes defining intelligence in terms of skill acquisition efficiency. His analysis calls for accounting for prior knowledge, available experience, and the difficulty of generalization. A high score tells us something about a skill; on its own, it does not adequately measure the ability to learn.

This distinction changes how a demonstration should be interpreted. A system that succeeds after extensive preparation and one that discovers a rule from a few examples may produce the same answer. Their final results can conceal differences in the resources required and the adaptation involved.

This approach does not demand learning without prior knowledge. It asks that this knowledge be made as explicit as possible. Comparing systems requires examining what they were given, what they had already encountered, and what they actually had to discover.

The article On the Measure of Intelligence explained introduces this proposal more gradually. The practical implication is to measure transfer alongside the result: keep familiar elements in place, change a relevant rule, and observe how the system adapts.

This definition is a research framework, rather than a universally accepted definition of intelligence. Nor does it provide, by itself, a proof that something is impossible for deep learning. Above all, it calls for care when interpreting performance: the breadth of skills already acquired should not be confused with the ease of acquiring new ones.

Generalizing beyond familiar similarities

Generalization is the ability to succeed beyond the training examples. However, novelty comes in different forms and degrees. Processing another photograph under familiar conditions does not require the same kind of transfer as applying a rule to a combination of objects never encountered before.

A random data split can leave substantial similarities between training and test sets. The model is then evaluated on new examples, but not necessarily on a new problem structure. To test more demanding transfer, entire environments, task families, or combinations can be reserved for evaluation.

In Shortcut Learning in Deep Neural Networks, initially published in 2020, Robert Geirhos and his coauthors describe decision rules that succeed in standard evaluations but transfer poorly to more challenging conditions. This phenomenon of statistical shortcut learning helps explain why a good score can coexist with practical fragility.

Imagine a document classifier that associates a page background with a category. It might perform well on existing archives and then fail after a change of scanner. To investigate this possibility, you would keep the content unchanged while altering the background, then run the reverse experiment.

The lesson concerns evaluation design: vary what should not change the answer. You also need to test changes that should alter it. A robust model must ignore some differences while remaining sensitive to others, so consistent answers alone are not enough.

Combining skills and sustaining reasoning

Being able to solve the parts of a problem does not guarantee being able to solve their combination. An assistant may retrieve useful information separately, then connect it incorrectly. Evaluation needs to distinguish access to knowledge, task decomposition, and the construction of the final answer.

Ofir Press and his coauthors examine this distinction in Measuring and Narrowing the Compositionality Gap in Language Models, submitted in 2022. In the models they studied, they observed a gap between success on individual subquestions and success on the combined question. They also showed that explicit decomposition methods could reduce this gap.

These historical findings do not automatically describe the models available in October 2026. They provide an analytical method that remains relevant: check intermediate results, how they are assembled, and the effect of structured assistance. An improvement produced by the protocol reveals both a usable capability and a dependence on that assistance.

This difficulty has practical consequences in a long task. An incorrect initial assumption can steer later steps, even when each appears locally coherent. When testing an agent, it therefore helps to maintain checkpoints covering the data, assumptions, and current state of the work.

A calculator or interpreter can make a particular operation more reliable. It does not validate the choice of operation or the relevance of the supplied data. Checks need to cover both individual steps and the final objective, using success criteria defined before execution.

Correlation, causality, and world models

Predicting an observation and predicting the effect of an intervention are different problems. An association between two variables is not enough to determine what would happen if one were deliberately changed. This distinction is particularly relevant to systems used to recommend actions.

In Towards Causal Representation Learning, published in 2021, Bernhard Schölkopf, Yoshua Bengio, and their coauthors connect causality with difficulties in transfer and generalization. They also highlight the challenge of discovering relevant causal variables from observations, rather than assuming those variables are already available.

Consider a hypothetical example: long delays and customer complaints may occur together in business records. That observation alone does not predict the effect of changing how the business operates. The analysis needs to consider shared contributing factors, the proposed mechanisms, and the conditions under which the data was generated.

A language model can help formulate these hypotheses or explain causal reasoning. That does not give it access to interventions missing from the evidence. A persuasive answer cannot replace identifying the mechanism or obtaining data that distinguishes between competing explanations.

World models offer a related direction: learning representations that can anticipate the consequences of actions. The challenge goes beyond generating a plausible description. To plan, a system needs predictions that are reliable enough over the relevant time horizons, and it needs to detect when its representations no longer fit the situation.

Producing accurate answers and recognizing uncertainty

Good writing is no guarantee of truth. When a system produces a reference, calculation, or explanation, the useful question is how the claim can be checked. A hallucination becomes particularly difficult to spot when it appears within otherwise correct material.

However, it would be wrong to claim that models cannot assess their own answers. In Language Models Mostly Know What They Know, published in 2022, Saurav Kadavath and his coauthors report encouraging results for self-evaluation and calibration in certain formats. They also find calibration difficulties on new tasks.

Calibration describes the correspondence between reported confidence and the actual frequency of success. It is measured across a collection of cases; it does not guarantee that a particular answer is correct. A method calibrated in one setting should therefore not be treated as validated in every other setting.

For work involving documents, you can require supporting passages and check that they substantiate the specific claim. For a calculation, you can inspect the inputs and repeat the operation independently. When information is missing, abstaining from an answer needs to be an acceptable outcome.

Asking the system to review its answer can be part of the process. For an important decision, also arrange a check that introduces distinct information: a primary source, an executable test, a formal constraint, or human review. Repeating the same question does not, by itself, provide independent validation.

What scaling establishes—and what it leaves open

Criticisms of deep learning need to be considered alongside findings that support increasing resources. In Scaling Laws for Neural Language Models, published in January 2020, Jared Kaplan and his coauthors describe empirical relationships between prediction loss, model size, data, and training computation.

These observations support investigating methods that can make use of additional resources. Their scope remains bounded by the measurements and conditions studied. Better prediction does not automatically demonstrate operational reliability, causal capabilities, or efficient adaptation to every new task.

It would be equally excessive to turn an old failure into a permanent ceiling. Data, training objectives, architectures, and inference procedures can change. Any claim about a current inability therefore needs to identify the system evaluated, the assistance allowed, and the date of the finding.

Disagreement may concern the nature of the changes needed. One approach seeks improvements within more general neural methods. Another prioritizes adding explicit mechanisms for search, representation, or verification. These directions are not always mutually exclusive in an implementation.

A scaling law should therefore be understood as an empirical relationship tied to a measurement, rather than a universal promise. For a practical decision, compare the improvement achieved with a fixed resource budget, then examine whether that improvement addresses the errors that actually matter to you.

Researchers do not all propose the same solution

The diagnoses overlap in places, but the research programs differ. François Chollet puts efficiency in acquiring new skills at the center of his proposal. Research on causal representations focuses more on mechanisms that would allow knowledge to transfer when conditions change.

Yann LeCun presents another direction in A Path Towards Autonomous Machine Intelligence, published in June 2022. This position paper proposes predictive world models, abstract representations, and hierarchical planning, among other elements. It describes a research program grounded in learning, rather than demonstrating a completed system with general intelligence.

Approach	Central question	Direction explored	Validation needed
Chollet’s framework	How efficiently can a new skill be acquired?	Measure experience, prior knowledge, and transfer	Success on unfamiliar tasks with explicitly stated resources
Causal representations	Which mechanisms remain relevant when conditions change?	Learn causal variables and relationships	Predictions tested against interventions or controlled changes
LeCun’s program	How can a system learn to predict and plan?	World models and hierarchical representations	Action consequences and plans evaluated in the environment
Scaling research	How does a metric change with resources?	Adjust models, data, and computation	Empirical relationships reproduced within their domain of validity

These positions do not form two uniform camps. A world model can be neural, and program search can be guided by a network. The guide to program synthesis explains this relationship between learning and searching for explicit solutions.

The experimental challenge is to establish what each component contributes, with which resources and on which tasks. An architectural proposal becomes convincing when it produces measurable transfer beyond the intuition that motivated it.

ARC: making novelty easier to observe

The official ARC-AGI overview connects the benchmark to measuring skill acquisition on unfamiliar tasks. It also makes the role of prior knowledge explicit. Its methodological value lies in asking what a system learns from the task it is given, rather than looking only at what it already knows how to do.

Interpreting an evaluation requires distinguishing the model from the setup around it. Additional examples, solution search, repeated attempts, or a selection procedure change the resources available. The result applies to that complete setup under its execution conditions.

A protocol should specify, in particular:

the data used to prepare the system;
the information accessible during evaluation;
the search and selection operations allowed;
the resources consumed to obtain the answer;
the conditions needed to reproduce the result.

These details are not grounds for dismissing systems that use tools. They make it possible to compare like with like and identify where an improvement comes from. The capabilities of the complete setup can be useful even when they do not come from the model alone.

No benchmark, however, covers every capability required in an organization or a physical environment. To connect a public measurement to your needs, how to read model rankings explains how to examine scope, possible contamination, and differences in evaluation protocols.

Evaluating these limits in real use

To decide whether a system is suitable, turn each concern into an observable test. Instead of asking whether it generally understands invoices, define the fields to extract, the ambiguities to flag, and the errors that must stop processing. The guide to evaluating AI agents develops this approach.

Start with representative cases, then create controlled variations. Change the presentation without changing the meaning; introduce an exception that changes the answer; remove an essential piece of information. This lets you distinguish sensitivity to formatting, difficulty applying a rule, and missing data.

For long tasks, measure complete success as well as the points where the process breaks down. Also check recovery after an error: does the system detect an inconsistent result, reconsider its assumption, and stop when evidence is missing? A correct final answer can conceal a path that is difficult to reproduce.

Next, compare several configurations on the same cases with an explicit budget. Rule-based extraction, a model on its own, and a system with document retrieval operate under different constraints. Relevant costs also include verification, rework, and undetected errors.

Execution traces can be inspected with observability tools such as Langfuse. They need to be connected to outcome criteria and failure analysis: recording the steps is not enough to establish whether those steps were correct.

Key takeaways

A score measures performance under given conditions; it does not, on its own, reveal learning efficiency.
Statistical shortcuts, composition, causality, and calibration require separate evaluations.
François Chollet, researchers studying causality, and Yann LeCun propose different frameworks that are sometimes compatible.
Data, computation, tools, and verification mechanisms are all part of the system being compared.
Decisions about deployment should rest on representative tests, controlled variations, and explicit stopping criteria.

Frequently asked questions

Can deep learning learn beyond its training examples?

Yes. Learning involves transferring something to new examples. The challenge is establishing how far that transfer extends when rules, combinations, or observation conditions change. An evaluation therefore needs to specify what kind of novelty it introduces.

Do François Chollet's criticisms mean LLMs are useless?

No. His distinction between acquired skills and learning efficiency allows us to recognize a system's usefulness without inferring general intelligence. A tool can deliver measurable value while still requiring extensive training and a narrowly defined setting.

Can a neural network represent an algorithm?

Yes, depending on its architecture and training conditions. That does not guarantee that training will discover the intended algorithm or that the network will execute it correctly on longer inputs. Representational capacity, learning, and reliable execution are separate questions.

Do hybrid systems solve the limitations of deep learning?

They can move some operations to components whose outputs can be checked, such as an interpreter or a constraint solver. However, the choice of program, input data, and validation criteria can still be wrong. The complete system needs to be evaluated.

How can you identify a lasting limitation of AI?

Repeated failure across several models and evaluation protocols is evidence, not a universal proof. A theoretical limitation requires explicit assumptions and a proof. For practical decisions, the relevant criterion remains the reliability observed under the intended operating conditions.

The limits of deep learning: what models struggle with, according to François Chollet and other researchers