Training compute: measuring the effort behind an AI model
Training compute is the total number of math operations needed to train an AI model, expressed in FLOP. It is the most widely used yardstick for comparing the scale of models, and even a legal criterion in the European Union.
Data as of October 22, 2026
Key figures
| Level | Advanced |
|---|
The definition
Training compute measures how much work computers had to do to build an AI model.
The unit is the FLOP, a floating-point operation: one addition or multiplication between two decimal numbers. During pre-training, each token of text flows through the network and the model then adjusts its parameters, which takes billions of operations per token. Summed over the whole run, the totals are so large they are written as powers of ten, for example 10^24 FLOP.
A common approximation, used among others by Kaplan and co-authors in their 2020 paper on scaling laws, puts the compute of a Transformer-style model at roughly 6 × number of parameters × number of training tokens.
A worked example
Take a hypothetical model with 10 billion parameters trained on 1 trillion tokens: 6 × 10^10 × 10^12 = 6 × 10^22 FLOP. That is more than 150 times below the 10^25 FLOP threshold that Article 51 of the EU AI Act uses to presume a general-purpose model carries systemic risk. A back-of-the-envelope calculation like this lets you place a model without knowing every detail of its training.
Why it matters
Compute is one of the three drivers of AI progress, alongside data and algorithms, according to the 2022 study by Jaime Sevilla and co-authors. They describe three eras: before 2010, training compute doubled roughly every 20 months, in line with Moore’s law; in the deep learning era it began doubling roughly every 6 months; then, from late 2015, a new class of large-scale models appeared. The research group Epoch AI maintains a public database of notable models and their estimated compute, one of the sources behind our AI in numbers page.
Compute also translates into cost, into the number of GPUs involved and into energy use, which makes it an economic and environmental issue as well.
Common myths
- “More compute guarantees a better model.” Data quality and how the budget is split between size and data matter just as much.
- “Compute stops once training ends.” Reasoning models also spend heavily at answer time: that is test-time compute.
- “The published figure is exact.” For many closed models, the numbers are outside estimates.
Related terms: scaling laws, GPU, pre-training and the AI Act.
Related terms
- Scaling laws · Measured relationships that predict how a model's performance improves with its size, its training data and its compute.
- GPU (graphics processing unit) · A chip built to run thousands of calculations in parallel, which became the engine for training and running AI models.
- Pre-training · The first training phase of a language model: it learns to predict the next piece of text across a huge corpus, with no specific task.
- Model parameters · The internal numbers a model tunes during training; how many it has indicates its size, not directly how good it is.
- EU AI Act · The European law that regulates AI systems according to their level of risk, with bans and transparency duties.
- Test-time compute · The computing power a model spends while answering, for instance to think for longer or to compare several candidate solutions.
Try it
Full glossary
Frequently asked questions
What is the difference between FLOP and FLOPS?
A FLOP (floating-point operation) counts operations, and a training total is expressed in FLOP. FLOPS, or FLOP per second, measures how fast a chip is. Dividing the first by the second, adjusted for real-world utilization, gives a rough idea of training time.
How is compute estimated when a lab publishes nothing?
Researchers work from whatever is available: parameter count, number of training tokens, or the hardware and duration announced. Such estimates, like those from Epoch AI, come with an uncertainty range worth keeping in mind.
Why does the EU AI Act care about FLOP?
Under Article 51 of the EU regulation, a general-purpose AI model is presumed to have high-impact capabilities, and therefore systemic risk, when its cumulative training compute exceeds 10^25 FLOP. That status brings extra obligations for the provider.
Data sources
Data as of October 22, 2026
Report an error
Does a figure, a date or a fact look wrong? Tell us: every report is checked.