Attention mechanism: how a model links words together
The attention mechanism is the technique that lets a language model weigh, for every word it processes, how relevant each other word in the text is. It is the heart of the Transformer architecture, and therefore of today's AI assistants.
Data as of October 22, 2026
Key figures
| Level | Intermediate |
|---|---|
| Origin | 2014 |
| Introduced by | Bahdanau, Cho et Bengio |
How it works
When you read “Mary lent her bike to Paul because he had missed his bus”, you know that “he” means Paul. To get there, your brain linked that word to another one earlier in the sentence. The attention mechanism does something similar: for each token (a word or part of a word), the model computes a relevance score against every other token, then blends their information, giving more weight to the most relevant ones.
Each word ends up with a representation enriched by its context. The word “bank” will not be represented the same way in “the river bank flooded” and “the bank approved the loan”, because attention links it to different neighbours.
Where the idea came from
The mechanism was popularised in 2014 by Dzmitry Bahdanau, Kyunghyun Cho and Yoshua Bengio in the paper “Neural Machine Translation by Jointly Learning to Align and Translate”. Their translation model could, for each word it produced, “search” the source sentence for the useful parts instead of squeezing everything into a single fixed-length vector. In 2017, “Attention Is All You Need” went a step further: the Transformer relies on attention alone, applied to the text itself.
Why it matters to you
Attention explains two things you notice when using an assistant. First, its ability to pick up a detail you mentioned much earlier in the conversation. Second, the existence of a context window: because every token is compared with all the others, compute and memory costs rise quickly with length. Techniques such as the KV cache avoid redoing some of the work, and research on long context aims to push the limit further.
The site’s context window tool shows you in practical terms how much text fits into a model’s working memory.
Common myths
- “Attention weights explain the model’s decisions.” They offer clues, but researchers debate how much they really explain: a model has many layers and attention heads interacting with one another.
- “The model pays equal attention to the whole context.” In practice, information buried in the middle of a very long document can be picked up less reliably.
- “Attention was invented in 2017.” It existed before; 2017 is when it became the sole pillar of the architecture.
Related terms
- Transformer · A neural network architecture built on attention, introduced in 2017, that underpins almost every large language model today.
- Context window · The maximum amount of text (in tokens) a model can read and produce at once, including its own answer.
- KV cache · Memory where a model keeps the work already done on previous tokens so it does not have to redo it for every new word.
- Long context · A model's ability to process very long inputs in one go, such as books or code repositories, and the techniques that make it possible.
- Token · A small chunk of text (a word, part of a word or a symbol) that an AI model reads and writes; the unit for limits and pricing.
Try it
Full glossary
Frequently asked questions
Is a model's attention like human attention?
Only by analogy. For the model it is a calculation of scores between token representations, not conscious focus. The word was chosen because it describes the effect well: zeroing in on what matters.
What is the difference between attention and self-attention?
In the original version, a translation model looked back at the source sentence while writing the translation. In self-attention, each word in a text looks at the other words in that same text. Transformers rely mainly on the latter.
Why does attention limit how long a text can be?
Every token is compared with every other token, so the number of comparisons grows much faster than the length of the text. Model makers use various optimisations to push the limit back, but it still weighs on compute and memory costs.
Data sources
Data as of October 22, 2026
Report an error
Does a figure, a date or a fact look wrong? Tell us: every report is checked.