Scaling laws: why AI models keep getting bigger
Scaling laws are measured relationships that predict how much an AI model improves when you increase its size, its training data and the compute spent on it. They largely explain the race toward ever-larger models since 2020.
Data as of October 22, 2026
Key figures
| Level | Advanced |
|---|---|
| Origin | 2020 |
| Introduced by | Jared Kaplan et al. |
What scaling laws say
Put simply: the bigger the model, and the more data and compute it is trained with, the better it gets, and that improvement follows a predictable curve.
In January 2020, Jared Kaplan and co-authors published “Scaling Laws for Neural Language Models”. They measured a language model’s loss, meaning its average error when predicting the next token, and found that it falls as a power law of three factors: the number of parameters, the amount of training data and the training compute. The trends held across more than seven orders of magnitude. A notable detail: the exact shape of the network, width or depth, made little difference within a wide range.
In practice, a power law shows up as a straight line on a log-log chart: every multiplication of the budget buys a steady drop in loss, but each step costs more in absolute terms.
Chinchilla: the 2022 correction
Kaplan’s team recommended training very large models on a relatively modest amount of data. In March 2022, Jordan Hoffmann and co-authors revised that conclusion in the Chinchilla paper: for the same compute, model size and number of tokens should scale equally. Their proof point was Chinchilla, a 70-billion-parameter model trained on four times more data than Gopher with the same compute, which outperformed Gopher (280 billion) and GPT-3 (175 billion) across a wide range of evaluations.
Why it matters
Scaling laws work as a planning tool: a lab trains small models, fits the curve, then predicts how a far more expensive model will perform before committing to it. They have justified massive investment in GPUs and data centers. They also shape practical choices: a smaller model trained for longer is cheaper to run afterward, which matters for products serving millions of people.
Common myths
- “Bigger is always better.” Only with an optimal budget: an under-trained large model can lose to a smaller one, as Chinchilla showed.
- “Lower loss means a smarter model.” Loss measures text prediction; some abilities emerge in a far less regular way.
- “Scaling is over.” The debate is real, but labs are now extending the idea to test-time compute.
Related terms: pre-training, training compute and parameters.
Related terms
- Training compute (FLOP) · The total number of arithmetic operations, measured in FLOP, needed to train an AI model.
- Model parameters · The internal numbers a model tunes during training; how many it has indicates its size, not directly how good it is.
- Training data · The examples (text, images, audio and more) an AI model learns from; they shape what the model knows and where it goes wrong.
- Pre-training · The first training phase of a language model: it learns to predict the next piece of text across a huge corpus, with no specific task.
- Test-time compute · The computing power a model spends while answering, for instance to think for longer or to compare several candidate solutions.
- Large language model (LLM) · A program trained on huge amounts of text to predict what comes next in a text; it is the engine behind AI assistants.
Try it
Full glossary
Frequently asked questions
Do scaling laws prove AI will keep improving forever?
No. They are regularities observed over a given range, not laws of nature. They predict lower error at predicting text, which does not automatically translate into useful skills, and nothing guarantees they hold beyond the scales that were measured.
What did the Chinchilla paper change?
Published in 2022 by Jordan Hoffmann and co-authors, it showed that for a fixed compute budget, model size and the number of training tokens should grow in equal proportion. Many large models of the time were therefore under-trained.
Are there scaling laws for reasoning?
Labs also describe gains from compute spent at answer time, for example by letting a reasoning model think longer. Those relationships are newer and less firmly established in public research than pre-training scaling laws.
Data sources
Data as of October 22, 2026
Report an error
Does a figure, a date or a fact look wrong? Tell us: every report is checked.