Token estimator

What is a token?

A language model does not read words the way we do. Before processing a text, it breaks it into tokens: chunks of words. A token can be a whole word, part of a word, a punctuation mark or even a space. Common words often fit in a single token, while rare, long or technical words are split into several pieces.

Tokens are the unit of measure of generative AI. Providers bill their models per token, and they express the size of the context window, the amount of text a model can take into account at once, in tokens too.

How the estimator works

Paste your text into the box above. The tool counts characters and words, detects the likely language, then applies an approximate rule for that language to estimate the token count. It does not use any tokenizer library (the software each provider uses to do the actual splitting): it relies on a heuristic, a quick and transparent estimation method.

You then get:

• the estimated cost of sending this text to 6 popular models, as input tokens;
• the estimated cost if a text of this length were generated as a reply, as output tokens, which are usually pricier;
• the share of each model’s context window this text would use.

Your text stays in your browser: nothing is uploaded, nothing is kept.

Why other languages often use more tokens than English

Tokenizers are built from huge collections of text in which English usually takes up a large share. Frequent English words therefore tend to map to a single token. In languages such as French or Spanish, accents, rich verb conjugations and sometimes longer words often lead to a finer split.

The upshot: to say the same thing, a French or Spanish text frequently needs more tokens than its English equivalent, so it costs a little more and fills the context window faster. The gap varies by model and by type of text, and the estimator accounts for it approximately.

Why it is only an estimate

Every model family has its own tokenizer. The same paragraph will not produce exactly the same token count with two providers, and code, tables, numbers or emoji are split differently again. The estimate gives you a ballpark good enough to make decisions, not an exact count. For a precise figure, providers offer their own counting tools, and their API (the interface that lets a program call the model) reports the tokens actually used after each request.

Practical uses

• Plan a budget: measure a typical prompt, then enter the numbers in the AI agent cost calculator to project a monthly bill.
• Check that a document fits: before pasting a long report into an assistant, see how much of the context window it takes.
• Trim a prompt: compare two versions of the same instruction and keep the shorter one if it works just as well.
• Compare models: differences in input and output prices show up immediately, and the model comparison tool then details their capabilities.

If a term is unfamiliar, the glossary explains the key AI concepts in plain English.