ARC-AGI explained: the test of AI intelligence

ARC-AGI is a family of tests that evaluates how an AI system learns to solve unfamiliar problems with limited information. Its early editions use colored grids; the interactive edition also requires systems to explore an environment and discover its rules. The benchmark provides a focused measure of adaptation, but cannot establish on its own that artificial general intelligence exists.

What ARC-AGI aims to measure

An AI system may pass a test because it encountered many similar problems during training. That result demonstrates a skill, but does not, by itself, explain how the skill was acquired. ARC-AGI aims to make this distinction observable.

In On the Measure of Intelligence, published in November 2019, François Chollet proposes defining intelligence in terms of the efficiency with which skills are acquired. His approach accounts for prior knowledge, available experience, and the difficulty of generalization. It therefore distinguishes final performance from the ability to learn when faced with an unfamiliar situation.

Here, generalization means identifying a rule and applying it beyond the examples provided. A transformation learned from one arrangement should still work when objects change position, size, or color, provided those variations preserve the problem’s underlying logic.

ARC-AGI does not ask systems to learn without any starting point. Its design explicitly assumes basic knowledge, particularly about objects, space, and quantities. The aim is to reduce the role of encyclopedic knowledge so that learning a rule specific to the task becomes easier to observe.

This also explains the benchmark’s scope: it examines a particular conception of intelligence, rather than an inventory of every human ability. Our article on François Chollet’s approach to measuring intelligence explores that distinction.

How the colored grids work

In the grid editions, you see examples pairing an input with an output. You then receive a new input and must construct the corresponding output. The overall instructions stay the same, but the transformation you need to discover changes from task to task.

The official ARC-AGI-1 repository describes rectangular grids represented by numbers associated with colors. It specifies that solving a task means producing the expected outputs with the correct dimensions. Every cell must match exactly.

Consider this made-up teaching example, intended to explain the principle, rather than reproduce an official task. The demonstrations contain solid shapes and hollow shapes. In every case, the output keeps only the hollow shapes, leaving them in their original positions.

For the new input, recognizing a familiar color is not enough. You need to identify the shapes, determine which ones have an empty interior, and remove the others while preserving the positions that matter.

A competing hypothesis might be to keep only the largest shapes. That would appear correct if every hollow shape in the demonstrations also happened to be among the largest. Another example could distinguish between these explanations; without one, you have to choose the hypothesis best supported by the observations.

The exercise therefore tests the choice of rule as well as its execution. A sound intuition followed by a single incorrect cell does not produce an exact solution. Conversely, a correct output does not automatically reveal the method used to obtain it.

Why a visually simple problem can be difficult

A simple visual format does not guarantee simple reasoning. A grid does not necessarily tell you what counts as an object, which color represents the background, or which property determines the transformation. Working these things out is part of the task.

Return to the teaching example. Before selecting the hollow shapes, a system must decide which cells belong together. Shapes that touch can make this segmentation harder, while a change in scale can invalidate a rule based on fixed dimensions.

Difficulty also comes from combining operations. Detecting an object, assigning it a property, selecting a transformation, and preserving particular spatial relationships are distinct steps. An early mistake can make the entire output inconsistent.

To analyze a task, you can separate out these questions:

• Which elements seem to stay the same between input and output?
• Which elements change consistently?
• Which hypothesis explains every available demonstration?
• Does that hypothesis require exceptions with no supporting evidence?

These questions offer a way to examine a task, rather than a universal solving algorithm. They show why a plausible description is insufficient: the rule must actually produce the correct grid.

This kind of analysis also helps distinguish different failures. A system may perceive objects incorrectly, choose the wrong rule, or execute an appropriate rule incorrectly. An overall percentage combines these difficulties without necessarily revealing which one caused a failure.

ARC-AGI-1, ARC-AGI-2, and ARC-AGI-3 are distinct tests

The ARC-AGI name covers several editions. The official ARC-AGI-1 overview dates its introduction to 2019. The first edition establishes the core idea: inferring an unfamiliar transformation from demonstrations.

Introduced in 2025, ARC-AGI-2 retains the grids, with particular emphasis on symbolic interpretation, combining rules, and applying them according to context. A mark can therefore serve a function within a task beyond its visual appearance.

ARC-AGI-3, available in 2026, introduces interactive environments. The system must act to gather information, understand the mechanics, and discover the objectives. It no longer simply receives all the demonstrations before answering.

Criterion	ARC-AGI-1	ARC-AGI-2	ARC-AGI-3
Format	Static grids	Static grids	Interactive environments
Initial information	Input and output examples	Input and output examples	Observations gathered through interaction
Required task	Infer a transformation	Infer rules and their interactions	Explore, learn, and plan
Observed outcome	Exact outputs	Exact outputs, with attention to efficiency	Success and efficiency of actions

These editions therefore do not form an interchangeable scale. The same percentage can refer to different tasks and assessment criteria.

The benchmark should also be distinguished from ARC Prize: the former defines an evaluation, while the latter organizes competitions and sets their participation conditions. Our ARC-AGI reference page brings together the editions and protocols to consult before comparing results.

With interaction, learning also means choosing actions

When demonstrations are provided in advance, the test taker does not have to decide how to gather information. In an interactive environment, that choice becomes central. An action can move the system toward its goal, but it can also help uncover a rule the system does not yet understand.

Imagine a teaching environment where a movement reveals that an area blocks passage. The system must incorporate that observation and change its route. Repeating the same movement without using the feedback suggests difficulty adapting, even if a long sequence of attempts eventually succeeds.

The ARC-AGI-3 scientific report, revised in April 2026, describes an evaluation based on abstract environments, objectives that must be inferred, and efficiency measured against human reference performance in terms of action counts. The path to a result therefore matters alongside the final state reached.

This brings the test closer to questions that arise with agents: what to observe, which action to try, when to revise a hypothesis, and how to retain useful information. It does not, however, turn an abstract game into a complete simulation of professional work.

A controlled environment makes it possible to isolate particular abilities. Real tasks add other constraints: ambiguous documents, unavailable tools, permissions, or the consequences of a mistake. The AGI and research section places these measurements within a broader set of approaches to intelligence.

How to read a score without overstating it

A score describes a system under a particular evaluation protocol. You need to know the edition, the tasks used, the permitted answers, and the available resources. A model name does not capture all those conditions.

For example, the official results page, accessed on October 24, 2026, lists an entry for GPT-6.1 Sol dated September 29, 2026: 98.5% on ARC-AGI-1, 94.2% on ARC-AGI-2, and 96.37% on ARC-AGI-3. These values come from separate evaluations; their numerical similarity does not make them equivalent.

This example illustrates why a historical result needs to retain its date. A snapshot taken when an edition launches does not necessarily describe performance several months later.

Before citing a result, check:

• The evaluation set: public, semi-private, or private tasks.
• The configuration: instructions, tools, and how answers are processed.
• The permitted attempts: a single answer and multiple proposed answers are different testing conditions.
• The resources: available compute, budget, and time.
• The source: an official measurement, an independent reproduction, or a reported result.

Public task sets make experimentation easier, but reusing them during development complicates the interpretation of progress. Benchmark contamination concerns exposure to evaluation data; a high score alone does not establish that contamination occurred.

For a closer look at interpreting results, Understanding model rankings explains the information that should accompany a results table.

Why compute and the complete system matter

A solution may involve more than generating an answer directly: it can use multiple hypotheses, a transformation program, checks, or a process for selecting among candidates. The underlying model therefore needs to be distinguished from the complete system taking the test.

The official model evaluation repository illustrates this experimental layer: running a benchmark requires code, configurations, and a measurement procedure. Knowing only the model provider is insufficient to reproduce a result.

Consider two hypothetical configurations. One quickly finds a correct rule; the other explores many possibilities before arriving at a solution. Their final success may look identical, but their costs and usefulness under a time limit differ.

Using additional compute is not inherently invalid. It simply needs to be visible. An improvement can come from a better method, more resources, or a combination of the two; the percentage alone cannot tell you which.

Monetary cost also requires careful interpretation. It depends on pricing and infrastructure, while elapsed time depends partly on parallel processing. These measurements illuminate different aspects of efficiency, but they do not replace an examination of the method.

For your own evaluation, define the constraints before measuring performance. Otherwise, you may end up comparing a restricted configuration with a system allowed far more attempts.

Passing ARC-AGI does not certify a system as AGI

An evaluation can be demanding while still covering only part of the picture. ARC-AGI alone does not assess social understanding, the full range of scientific knowledge, reliability in business software, or behavior when requests conflict.

Its design also involves choices: the types of problems included, assumed prior knowledge, the interface, and the criteria for success. Those choices make evaluation possible, but they also set boundaries on what the results justify claiming.

The human reference point deserves similar care. Showing that people can solve a task does not mean every person will solve it immediately. A comparison should specify the participants, attempts, and resources, rather than treating it as an abstract contest between a machine and all of humanity.

A high score therefore provides information about the abilities being evaluated. It offers neither a comprehensive measure of intelligence nor an automatic date for the arrival of AGI. Conversely, failure on grids does not erase all of a system’s practical skills.

To choose a tool for professional use, start with your tasks: the quality required, acceptable errors, cost, and necessary oversight. The Evaluating an AI agent guide explains how to build this complementary assessment. ARC-AGI then serves as a research signal to weigh against results in your own context.

Key takeaways

• ARC-AGI examines how systems learn unfamiliar rules from limited information within a defined scope.
• The grid editions and the interactive edition evaluate different behaviors.
• A score needs to retain its date, protocol, task set, and computing conditions.
• Success provides focused evidence of capability, without independently certifying artificial general intelligence.

Frequently asked questions

What does ARC-AGI stand for?

ARC stands for Abstraction and Reasoning Corpus. The name ARC-AGI connects that corpus to the evaluation of artificial general intelligence, but passing it does not certify a system as AGI.

Can I try ARC-AGI without knowing how to code?

Yes. Interfaces linked from the official pages let you solve grids manually or play the interactive environments. You need programming skills to build a system that takes the evaluations automatically.

Is ARC-AGI an IQ test for AI?

The comparison helps explain the idea of tackling unfamiliar problems, but it only goes so far. An ARC-AGI percentage cannot be converted into a human IQ score: the tasks, learning conditions, and available resources differ.

Why do ARC-AGI scores differ across publications?

Publications may use different editions, task sets, or system configurations. The computing budget and number of permitted answers also matter. Comparing results therefore requires more than a model name and a percentage.