#Reranking & filtering

The one sentence: retrieval is optimised to not miss things, reranking is optimised to put the right one first, and asking one component to do both is why so many pipelines are mediocre at each.

If you add one thing to a working RAG pipeline, add a reranker. It is the change with the most consistent measured benefit and the least architectural disruption.

#1 · Diagram

   TWO STAGES, TWO OBJECTIVES

   STAGE 1 - RETRIEVE            optimise RECALL
   ------------------            get the right passage into the pool AT ALL
   bi-encoder over the corpus    cheap: one vector comparison per document
   top 50                        precision does not matter here

                │
                ▼

   STAGE 2 - RERANK              optimise PRECISION
   ---------------               put the right passage FIRST
   cross-encoder over 50 pairs   expensive: a model pass per (query, passage)
   top 5                         only affordable because 50 << corpus


   WHY A CROSS-ENCODER IS BETTER, and why you cannot use it alone

   bi-encoder     embed(query) · embed(passage)     -> vectors never meet
                  precomputable, one comparison      -> scales to millions
                  cannot model INTERACTION

   cross-encoder  model(query + passage together)   -> full attention across both
                  nothing precomputable              -> one pass PER PAIR
                  100x more accurate on hard cases   -> 100x too slow for a corpus

BM25 is three terms multiplied together, and you can break each one by hand. Type documents, drag k1 and b, and watch the two failures that put a second stage in the pipeline.

#2 · Design

Intermediate — the options, in ascending cost.

Depth-versus-latency is the parameter that matters. Reranking cost is linear in shortlist size, so the retrieve-50-rerank-5 shape is a direct trade:

   retrieve 20, rerank 5   fast, but if the answer was at rank 35 it is gone
   retrieve 50, rerank 5   the usual sweet spot
   retrieve 200, rerank 5  better ceiling, 4x the rerank cost

Choose it by measuring retrieval recall at each depth on your eval set. If recall@50 and recall@200 are both 0.94, reranking 200 buys nothing but latency.

Advanced — filtering is reranking's neglected sibling and does work no model can. Some reordering is not a relevance judgement at all:

Deduplication before reranking is a genuinely cheap win. Near-duplicate chunks — from overlap, or from the same content in several documents — routinely occupy multiple top-k slots and crowd out the passage that would have completed the answer. Collapsing near-duplicates by content hash or high mutual similarity costs nothing and frequently improves end-to-end quality more than a reranker upgrade.

#3 · Flow

graph TD
  A[Query] --> B[Retrieve top-N<br/>bi-encoder, cheap]
  B --> C[Hard filters:<br/>permissions, date, source]
  C --> D[Deduplicate near-identical chunks]
  D --> E{Enough candidates left?}
  E -->|no| F[Widen N, or relax soft filters<br/>never relax permissions]
  F --> B
  E -->|yes| G[Rerank with cross-encoder]
  G --> H[Take top-k]
  H --> I{Top score below threshold?}
  I -->|yes| J[Treat as no relevant context<br/>-> corrective RAG path]
  I -->|no| K[Send to the model]

Filters before the reranker, always. Reranking a document the user may not see is wasted compute at best and, if a bug lets it through, a leak. Hard filters cost nothing and shrink the expensive stage.

I connects this page to corrective RAG. A reranker gives you a calibrated-ish relevance score, which is the natural signal for deciding there is nothing worth answering from.

#4 · UML — the two stages

sequenceDiagram
    participant Q as Query
    participant V as Vector index
    participant F as Filters
    participant X as Cross-encoder
    participant M as Model

    Q->>V: embed, search top-50
    Note over V: one vector comparison per document.<br/>Cheap enough for millions.
    V-->>F: 50 candidates
    F->>F: permissions, recency, dedupe
    F-->>X: ~40 survivors
    loop each (query, passage) pair
        X->>X: full attention across both texts
    end
    Note over X: 40 model passes. Only affordable<br/>because it is 40, not 4 million.
    X-->>M: top 5, reordered
    M-->>Q: grounded answer

#5 · Example

def retrieve_and_rerank(index, reranker, query, *, user, depth=50, k=5,
                        min_score=0.3):
    """Two-stage retrieval with filtering between the stages.

    Order matters: filter before reranking, because reranking is the expensive
    stage and there is no reason to spend it on documents that will be dropped
    -- or worse, that the user is not permitted to see.
    """
    candidates = index.search(query, depth)

    # Hard filters. Permissions are a correctness requirement and are applied
    # first; a relevance model must never be able to override them.
    candidates = [c for c in candidates if user.may_read(c.source)]
    candidates = deduplicate(candidates)

    if not candidates:
        return []

    scores = reranker.predict([(query, c.text) for c in candidates])
    ranked = sorted(zip(candidates, scores), key=lambda p: p[1], reverse=True)

    # A relevance floor turns the reranker into a decision, not just an order.
    # Without it the pipeline always returns its five best guesses, however bad.
    return [c for c, s in ranked[:k] if s >= min_score]


def deduplicate(candidates, threshold=0.95):
    """Collapse near-identical chunks before they consume top-k slots.

    Overlap during chunking and duplicated content across documents both
    produce near-copies. Three copies of one passage occupying three of five
    slots is a quality loss that no reranker can repair, because from its point
    of view all three genuinely are relevant.
    """
    kept = []
    for candidate in candidates:
        if all(similarity(candidate.text, k.text) < threshold for k in kept):
            kept.append(candidate)
    return kept

Choosing the shortlist depth by measurement, not by habit:

def recall_by_depth(index, eval_set, depths=(10, 20, 50, 100, 200)) -> dict:
    """Where does retrieval recall stop improving?

    Reranking cost is linear in depth, so paying for 200 when recall plateaus
    at 50 is pure latency. This is a ten-minute measurement that sets a
    parameter people otherwise copy from a tutorial.
    """
    out = {}
    for depth in depths:
        hits = sum(
            any(c.id in item.relevant_ids for c in index.search(item.query, depth))
            for item in eval_set
        )
        out[depth] = round(hits / len(eval_set), 3)
    return out

#6 · Depth — the senior layer

Reranker scores are not probabilities, even when they look like them. A sigmoid output of 0.7 does not mean 70% relevant, and the distribution shifts with query length and domain. If you use a threshold — and the corrective-RAG path needs one — calibrate it on your data, and re-calibrate whenever the reranker changes. A threshold copied from a model card is a guess wearing a number.

Where reranking does not help. If your retrieval recall at the shortlist depth is already near 1.0 and the right passage is already at rank 1, a reranker adds latency and nothing else. That is rare but real for small, well-structured corpora with distinctive vocabulary. Measure first — this is the counter-case to "always add a reranker", and knowing it exists is the difference between a rule and a judgement.

ColBERT sits between the two stages and is worth understanding as a category. It keeps a vector per token and scores by MaxSim, so document token vectors are precomputable — giving much of a cross-encoder's precision at closer to a bi-encoder's cost. The trade is storage: a vector per token is far larger than a vector per chunk. It is covered in its own right in the ColBERT material of the retrieval module.

Reranking is where hybrid retrieval resolves. BM25 and dense retrieval produce two ranked lists whose scores are not comparable. Reciprocal Rank Fusion merges them by rank, and a reranker then produces a single, consistent ordering over the fused pool. That is the standard production shape: hybrid retrieve, fuse, dedupe, rerank, threshold.

#7 · From each seat

#8 · Interview questions

#Stop condition

You are done when you can:

1. state the two stages' different objectives,
2. explain why a cross-encoder is more accurate and unusable alone,
3. justify a shortlist depth from measurement,
4. say why filtering and deduplication come first, and
5. name the case where a reranker is not worth adding.

#Sources worth reading