Reranking: getting the right passages to the top
Reranking is a second search stage: after a fast first pass selects candidates, a more precise model rereads each candidate against the query and recomputes the order of relevance. It is one of the most effective levers for improving a RAG pipeline.
Data as of October 22, 2026
Key figures
| Level | Advanced |
|---|---|
| Origin | 2019 |
| Introduced by | Nogueira & Cho |
Definition
Put simply: the first search casts a wide, fast net; reranking then sorts the catch carefully. This is known as two-stage retrieval, or retrieve then rerank.
The first stage uses precomputed embeddings, keywords, or both through hybrid search. It is fast but approximate: the query and the documents never “see” each other directly. A reranker, often a cross-encoder, takes the query and a passage in the same input and outputs a relevance score. It picks up nuances (negation, the level of detail requested, context) that vector similarity misses.
Historical markers
Reordering results is an old idea in information retrieval. Its application to modern language models was popularized by Nogueira and Cho’s paper, posted on arXiv in January 2019, in which a BERT model used as a reranker took first place on the MS MARCO leaderboard. The BEIR benchmark (2021) later showed that reranking models are among the best at generalizing to new domains, at the price of high compute cost.
A concrete example
A law firm queries its knowledge base: “Notice period when a tenant terminates a commercial lease.” Vector search returns twenty similar passages, several about residential leases and one about termination by the landlord. The reranker rereads each passage alongside the question and promotes the one that specifically covers tenants and commercial leases. The model in a RAG pipeline then gets the top three passages instead of twenty approximate ones: a more accurate answer and a shorter prompt.
Why it matters
- Precision: Anthropic reports, on its September 2024 test sets, that adding reranking to its method cut retrieval failures substantially further.
- Cost: sending fewer but better passages reduces the tokens consumed.
- Tradeoff: every reranked candidate adds compute, and therefore latency.
Common misconceptions
- “A good embedding model is enough.” The two stages do different jobs: recall first, precision second.
- “More candidates is always better.” Past a certain point, the gain is outweighed by latency.
Related terms: semantic search, hybrid search, RAG, embedding.
Related terms
- Hybrid search · Search that combines exact keyword matching with meaning-based (embedding) search, then merges both result lists into a single ranking.
- RAG (retrieval-augmented generation) · A technique that first retrieves relevant passages from your documents, then hands them to the model so its answer is based on them.
- Semantic search · Search that finds content by meaning rather than exact words, by comparing numerical representations of the texts.
- Embedding · A list of numbers (a vector) that captures the meaning of a text or image, so content can be compared by how close in meaning it is.
- Latency · The waiting time between sending a request to an AI model and receiving its answer, from the first word through to the last.
Full glossary
Frequently asked questions
What's the difference between a bi-encoder and a cross-encoder?
A bi-encoder turns the query and each document into vectors separately, which makes searching millions of documents very fast. A cross-encoder reads the query and a document together, which is far more precise but too expensive for the whole index, so it is reserved for the top candidates.
How many candidates should I rerank?
There is no universal number. In its September 2024 Contextual Retrieval study, Anthropic retrieves 150 passages and keeps 20 after reranking. The right tradeoff depends on your acceptable latency and budget.
Can an LLM be used as a reranker?
Yes: you can ask a language model to score or order a list of passages. It is flexible, but usually slower and more expensive than a dedicated reranking model.
Can reranking find a document that wasn't retrieved?
No. It only reorders what the first stage hands over. If the right passage was never retrieved, improve the initial search, for example with hybrid search.
Data sources
Data as of October 22, 2026
Report an error
Does a figure, a date or a fact look wrong? Tell us: every report is checked.