Prompt caching: don't pay twice to read the same thing
Prompt caching lets an AI model reuse the work it has already done on the start of a request, when that start is identical from one call to the next. The result: repeated parts are billed at a much lower rate and the answer starts sooner.
Data as of October 22, 2026
Key figures
| Level | Intermediate |
|---|
The principle
On every call, a model re-reads everything you send: the system prompt, documents, history, question. If the first few thousand tokens are identical from one call to the next, redoing all that computation is wasteful. Caching keeps the result of that processing in memory (in practice, KV cache tensors, according to OpenAI’s documentation) and reuses it as long as the prefix, meaning the start of the request, stays exactly the same.
A concrete example
An online store runs an assistant that answers customers from a long product guide placed at the top of every request. Without caching, that guide is re-read and billed at full price for every question. With caching, at Anthropic for example, according to its documentation:
- the first request writes the guide to the cache, at 1.25 times the normal input price;
- later requests read it from the cache at 0.1 times that price, as long as the cache hasn’t expired.
From the second question asked within the validity window, it already pays off: 1.25 + 0.1 = 1.35 times the price, versus 2 times without caching. Over hundreds of questions, the saving on the repeated part approaches 90%. To see the effect on your own usage, try our agent cost calculator.
Why it matters
Agents and document assistants keep sending the same bulky context. Caching lowers both the effective price per million tokens you pay and the latency, the delay before the answer starts. It is one of the easiest cost levers to pull.
Common mistakes and myths
- “Similar content is close enough.” The prefix must match exactly: changing a single word near the start (today’s date, for instance) invalidates the cache for everything after it.
- “Order doesn’t matter.” Put stable content (instructions, documents) first and whatever changes (the user’s question) last.
- “Caching is always free.” Writing to the cache can cost more than a normal read; if the content is never reused, you pay more.
Related terms: system prompt, KV cache, price per million tokens, context window.
Related terms
- Price per million tokens · How AI APIs bill usage: one rate per million tokens the model reads as input, and another, usually higher, per million it generates.
- KV cache · Memory where a model keeps the work already done on previous tokens so it does not have to redo it for every new word.
- System prompt · Background instructions given to a model before the conversation starts (role, tone, rules) that shape every answer that follows.
- Latency · The waiting time between sending a request to an AI model and receiving its answer, from the first word through to the last.
- Context window · The maximum amount of text (in tokens) a model can read and produce at once, including its own answer.
Try it
Full glossary
Frequently asked questions
Does the cache store my answers and replay them?
No. It keeps the result of processing the start of the prompt, not the answers. The model still generates a fresh response; it just skips re-reading and recomputing the shared part.
Do I need to turn caching on?
It depends on the provider. At OpenAI, it is enabled by default on supported models. At Anthropic, you turn it on with a cache_control parameter, either automatically for the whole request or at specific points.
How long does the cache last?
At Anthropic, the default lifetime is 5 minutes, refreshed for free each time it is reused, with a one-hour option that costs more to write. Other providers have their own rules, set out in their documentation.
Is there a minimum size?
Yes. Below a certain number of tokens, which varies by model, the prompt is not cached, and no error is returned. Short prompts therefore don't benefit from the discount.
Data sources
Data as of October 22, 2026
Report an error
Does a figure, a date or a fact look wrong? Tell us: every report is checked.