RAG: when AI checks your documents before answering
RAG (retrieval-augmented generation) first finds the relevant passages in a collection of documents, then hands them to the model so it writes its answer from them. It is the technique behind most assistants that can “answer from your files”.
Data as of October 22, 2026
Key figures
| Level | Intermediate |
|---|---|
| Origin | 2020 |
| Introduced by | Lewis et al. |
The idea in one sentence
Instead of relying on whatever the model memorized during training, you bring it the right pages at the right moment. RAG works in two steps: retrieve the relevant passages, then generate an answer grounded in them. The name comes from the paper by Patrick Lewis and co-authors, posted to arXiv in 2020.
How it works in practice
Picture a small company that wants an internal assistant able to answer HR policy questions.
- Preparation: the documents (employee handbook, leave policy, memos) are split into pieces, a step called chunking. Each piece is turned into an embedding, a list of numbers that captures its meaning, and stored in a vector database.
- Question: an employee asks “How many days off do I get when I move house?”. The question is also converted into an embedding, and the system retrieves the pieces closest in meaning.
- Answer: those passages are inserted into the prompt with an instruction such as “Answer only from the excerpts provided and cite them.” The model then writes an answer based on the actual policy.
Why it matters
Every model has a knowledge cutoff and knows nothing about your private documents. RAG fixes both gaps without retraining anything: updating the document store is enough. It also lets the system cite its sources, which makes answers easier to check and helps limit hallucinations. And because only the useful passages are sent, it saves room in the context window and money on the bill. To see how much text a model can read at once, try our context window tool.
Common mistakes and myths
- “With RAG, the AI can’t be wrong.” Quality depends heavily on retrieval: if the right passage isn’t found, the answer will be incomplete or wrong.
- “Just drop in your PDFs.” Badly split, outdated or contradictory documents produce inconsistent answers. Maintaining the document store matters as much as the model.
- “RAG always means vectors.” Retrieval can also use keywords, or a hybrid of both.
- “Long context windows make RAG obsolete.” They reduce the need for small collections, but for large document bases, selecting passages is still cheaper and often more accurate.
Related terms: embedding, vector database, chunking, semantic search, grounding.
Related terms
- Embedding · A list of numbers (a vector) that captures the meaning of a text or image, so content can be compared by how close in meaning it is.
- Vector database · A database that stores embeddings and quickly finds the ones closest to a query, which means the most relevant pieces of content.
- Semantic search · Search that finds content by meaning rather than exact words, by comparing numerical representations of the texts.
- Chunking · Splitting long documents into small, coherent passages so they can be indexed and only the useful pieces are sent to the model.
- Grounding · Making an AI base its answer on supplied sources (web results, documents, databases) rather than only on what it memorized in training.
- Hallucination · An AI answer that sounds confident and plausible but is false or made up: a figure, a quote, a source or a reference.
Try it
Full glossary
Frequently asked questions
What's the difference between RAG and fine-tuning?
RAG gives the model documents at question time without changing it. Fine-tuning changes the model itself through extra training. For information that changes often, RAG is usually simpler: you just update the documents.
Does RAG eliminate hallucinations?
It often reduces them but does not eliminate them. If retrieval brings back the wrong passage, or the model misreads the right one, the answer can still be wrong. Showing the cited sources lets users check.
Do I need a vector database for RAG?
Not necessarily. For a handful of documents, simple keyword search or even pasting everything into the context can work. A vector database becomes useful once your content exceeds what the model can read in one go.
Where does the term RAG come from?
It was popularized by the paper “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks”, posted to arXiv in May 2020 by a team led by Patrick Lewis. The approach then spread widely with large language models.
Data sources
Data as of October 22, 2026
Report an error
Does a figure, a date or a fact look wrong? Tell us: every report is checked.