Rate limits: the ceiling that controls access to AI models
A rate limit is a cap set by a provider on how much an account can send within a given time window. Go over it and further requests are refused until capacity builds back up.
Data as of October 22, 2026
Key figures
| Level | Intermediate |
|---|
What a rate limit measures
When you use a model through an API, every request uses up part of the provider’s capacity. To prevent abuse and share resources fairly, providers set caps per account. At Anthropic, according to the official documentation, Messages API limits are measured in:
- requests per minute (RPM);
- input tokens per minute (ITPM);
- output tokens per minute (OTPM).
These limits depend on a usage tier that changes with the account’s history. Anthropic says it uses a token bucket algorithm: capacity refills continuously rather than resetting at fixed times. A burst of requests can therefore trigger errors even when the per-minute average stays under the cap.
A concrete example
You launch an agent that has to summarise 500 product pages. Your script fires off every request at once: the first ones go through, then the API starts returning 429 errors. Good practice is to pace your calls, retry refused requests after an increasing delay, and read the response headers that show how much quota is left. To estimate the token volume and cost of a job like this in advance, try our agent cost calculator.
Why it matters
AI agents often chain many calls to complete a single task. Without proper limit handling, an agent can stop halfway through its work. Limits also protect your budget: they stop a badly written loop from sending thousands of requests in a few minutes.
One tip documented by Anthropic: for most of its models, tokens read from the cache do not count towards the input-token limit. Prompt caching therefore increases your effective throughput without changing tier.
Common mistakes and myths
- “A 429 means my key has been blocked.” Usually you just need to wait. An invalid API key returns different errors.
- “Just retry immediately in a loop.” That makes the overload worse. Space out your attempts.
- “Everyone gets the same limits.” They vary by provider, model, account tier and plan.
- “A per-minute limit lets me send everything in one second.” Bursts can be refused, because quotas are often enforced over shorter intervals.
Related terms: API, API key, price per million tokens, free vs paid AI.
Related terms
- API · An interface that lets one program send requests to a service, such as an AI model, and get its responses back automatically.
- API key · A secret code that identifies your account to an API and is used to bill your usage: never share it or publish it anywhere.
- Token · A small chunk of text (a word, part of a word or a symbol) that an AI model reads and writes; the unit for limits and pricing.
- Prompt caching · Reusing the start of a prompt the model has already processed, so repeated requests run faster and the reused part is billed at a lower rate.
- Price per million tokens · How AI APIs bill usage: one rate per million tokens the model reads as input, and another, usually higher, per million it generates.
- Free vs paid AI · How free and paid AI assistant plans differ: usage limits, available models, features and privacy options.
Try it
Full glossary
Frequently asked questions
What does a 429 error mean?
It is the HTTP status “Too Many Requests”, defined in RFC 6585, published in April 2012. It means too many requests were sent in a given amount of time. The response may include a Retry-After header telling you how many seconds to wait before trying again.
What should I do when I hit the limit?
Wait and retry, ideally with increasing delays between attempts rather than an instant loop. Honour the retry-after header when it is present. If it keeps happening, smooth out your traffic, send fewer tokens or ask for a higher limit.
Is a rate limit the same as a spend limit?
No. A rate limit governs volume over a short window, often a minute. A spend limit caps the amount billed, for example over a month. At Anthropic, reaching the monthly spend cap also returns a 429 error, but without a retry-after header, and retrying will not help until access resumes.
Why do consumer chat apps limit messages too?
For the same underlying reason: computing capacity is shared and expensive. Free plans generally have lower caps than paid plans. The exact rules vary by service and change regularly.
Data sources
Data as of October 22, 2026
Report an error
Does a figure, a date or a fact look wrong? Tell us: every report is checked.