Fusing Multiple Retrievers

A billing assistant searched for duplicate charge on invoice INV-88213. Keyword search returned the passage quoting the invoice number first — BM25 needs the term overlap it has. Vector search returned the policy paragraph "charged twice for the same invoice, credited within five days" first — the paraphrase shares no terms with the query but every meaning.

Each retriever is right about something the other misses:

The scores cannot be averaged. BM25 returns unbounded idf-weighted sums whose magnitude depends on corpus statistics; cosine similarity lives in a bounded range. Averaging numbers from those two scales is arithmetic without meaning — and the two retrievers do not even agree on how to spend their top ranks.

query --> [keyword search] --> ranked list A --+
      |                                       +--> [ fusion ] --> final order --> rerank / prompt
      +--> [vector search] --> ranked list B --+

Reciprocal rank fusion sidesteps the scale problem by throwing the scores away. Keep each document's position in each list, convert position to 1 / (k + rank), and sum across lists. Ranks are comparable across any two retrievers by construction — both are just "how high up did you put it" — which is why fusion needs no calibration step at all. The disagreement itself is the asset: Hybrid Search Basics covers why the two channels complement each other, and Reranking with Cross-Encoders covers what to do with the fused list afterwards.

The formula has one input per list — the document's 1-based rank — and one constant:

RRF(d) = sum over lists  1 / (k + rank_of_d_in_that_list)      k is usually 60

A document missing from a list contributes nothing; there is no penalty and no credit for absence.

from collections import defaultdict

def rrf(rankings: list[list[str]], k: int = 60) -> dict:
    scores = defaultdict(float)
    for ranked in rankings:                              # each list ordered best first
        for rank, doc in enumerate(ranked, start=1):     # 1-based: the best document is rank 1
            scores[doc] += 1 / (k + rank)
    return dict(sorted(scores.items(), key=lambda kv: kv[1], reverse=True))

keyword = ["d7", "d2", "d9", "d4"]
vector  = ["d21", "d7", "d2", "d4"]
print(list(rrf([keyword, vector])))

Output: ['d7', 'd2', 'd4', 'd21', 'd9']

Work the arithmetic for each document once:

Two readings matter more than the totals. d4 sits mid-list in both searches and beats d21, which was first in one of them — appearing consistently across retrievers outweighs appearing brilliantly once, and that is exactly what the 1/(k + rank) shape encodes. d7 wins by being near the top twice, not by any single great position. Note also what never enters the sum: the magnitude of a BM25 score, the size of a cosine, and any need to decide whether 12.4 "means the same as" 0.71.

start=1 in the loop is the line to defend in review. Zero-based ranks would make the top document worth 1/k instead of 1/(k+1), and every list would gain a phantom half-rank of credit that shifts scores as list length changes.

k is not a smoothing detail — it decides which kind of document wins. Run the same two lists with k=1:

With k=1, d21's single rank-1 appearance outweighs d4's two rank-4s and the order flips. A small k rewards one-list champions; a large k flattens contributions toward "how many lists included it". Neither is correct in the abstract — pick the value that scores best on your labeled queries and re-check it when you add or remove a retriever. 60 is a starting default from the original paper, not a constant of nature.

Fusion is also only as debuggable as the record you keep. Store the per-list rank beside the fused score:

debug = {"doc": "d4", "rrf": 0.03125, "ranks": {"keyword": 4, "vector": 4}}

When the final answer is wrong, that line tells you whether the keyword channel or the vector channel carried the passage — the difference between tuning a retriever and tuning blind.

Failure modes worth checking on every fusion deployment:

• Fusing near-identical lists — two retrievers returning the same order add no coverage; measure list overlap before claiming a fusion win (Hybrid Search Basics).
• Thresholding fused scores — 1/(k + rank) sums are not probabilities; they move whenever k or the number of lists changes.
• Unequal list lengths — absence contributes zero, which is fine, but a list truncated at 5 has far less influence over the outcome than one that runs to 30.
• Fusion as a repair for a broken retriever — a noisy list pushes its documents into the middle of the fused order; fusion merges rankings, it does not clean them.
• No provenance in the logs — without per-retriever ranks, a wrong fused answer cannot be attributed to a channel.