Test-time compute: when AI thinks longer to answer better
Test-time compute is the computing power a model uses at the moment it answers, as opposed to during training. Giving it more, so it can reason for longer or try several approaches, often improves its answers to hard problems, at the cost of more waiting time and a higher bill.
Data as of October 22, 2026
Key figures
| Level | Advanced |
|---|
What are we talking about?
For years, models improved mainly by scaling up training compute: more data and more parameters, following scaling laws. Test-time compute opens a second lever: spending more computation on each answer, during inference.
The main techniques:
- Reasoning for longer. The model writes an extended chain of thought, backtracks and double-checks its working.
- Sampling several answers. You generate multiple solutions and keep the most frequent one (majority voting) or the one a verifier scores highest.
- Searching with a verifier. A separate model grades each intermediate step and guides a search through possible paths.
- Revising. The model rereads and corrects its own answer over several passes.
Origins and key research
The paper “Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters” (Snell et al., August 2024) showed that how well these strategies work depends heavily on question difficulty, and that allocating compute wisely lets a small model match a much larger one on some tasks for the same budget. That same year, OpenAI introduced its o1 series, trained with reinforcement learning to reason before answering. The reasoning models that followed all build on this principle.
A concrete example
You ask an assistant to find the mistake in a loan repayment schedule. In fast mode it replies instantly, and sometimes misses. In thinking mode it recalculates each row, spots the inconsistency and explains the fix, after several seconds or even minutes. If you use an API, those thinking tokens show up on your invoice: our agent cost calculator helps you estimate how price per token affects your budget.
Common misconceptions
- “More thinking is always better.” Past a certain point, gains level off, and a model can even talk itself into confusion on a simple question.
- “The visible reasoning is exactly what the model thinks.” What the interface shows is often a summary, and a chain of thought is not guaranteed to be a faithful account of the model’s internal computation.
- “It’s free for users.” Consumer plans include thinking modes but often cap how much you can use them; through an API you pay for it directly.
Related terms
- Reasoning model · A language model trained to think step by step before answering: slower and pricier, but stronger at maths, code and logic.
- Chain of thought · Getting a model to write out intermediate reasoning steps before its final answer, which helps on multi-step problems.
- Inference · The moment a trained model is actually used to produce an output: every question you send to an AI triggers an inference run.
- Scaling laws · Measured relationships that predict how a model's performance improves with its size, its training data and its compute.
- RLVR (reinforcement learning with verifiable rewards) · Reinforcement learning in which the reward comes from an automatic check, such as a correct answer or passing code tests, not from a judgement.
- Price per million tokens · How AI APIs bill usage: one rate per million tokens the model reads as input, and another, usually higher, per million it generates.
Try it
Full glossary
Frequently asked questions
Why does a reasoning model cost more for the same question?
Because it first generates many thinking tokens, often hidden, before the final answer. Those tokens are usually billed as output tokens. The higher the reasoning effort you request, the higher the bill and the longer the wait.
Should I always turn reasoning effort up to the maximum?
No. For rewording, translation or a simple factual question, the gain is small or nil. Keep high settings for multi-step problems: maths, code, analysing complex documents, planning.
Can test-time compute replace bigger models?
Only partly. It helps a model make better use of what it already knows, but it does not add knowledge. A model that is weak on a topic will not master it just by thinking longer.
Data sources
Data as of October 22, 2026
Report an error
Does a figure, a date or a fact look wrong? Tell us: every report is checked.