Hybrid Search Basics

Run a keyword retriever and a vector retriever over the same query and they disagree — which means one of them is wrong about something specific. Knowing what each scorer is blind to is the entire design problem.

BM25 sums a per-term score over the query:

                       ┌──► BM25 over inverted index ──► ranked K ┐
                       │                                          │
query ─────────────────┤                                          weighted fusion ──► top-k
                       │                                          ▲
                       └──► embed ──► ANN index ──► ranked V ─────┘

Three knobs, three behaviors: IDF makes a rare term outweigh the; term frequency f grows but saturates at k1, so the fifth occurrence adds little; b normalizes by document length so long documents stop winning by size alone. Under all of it sits exact token matching — a term is either in the inverted index or it is not.

The vector scorer never sees tokens. It embeds the query, probes the ANN index, and returns documents whose vectors point in a similar direction (ANN Index Types). It has no notion of spelling, so E_CONN_RESET can sit near connection reset by peer — and SKU-7741-BLK can sit near SKU-7741-RED, which is exactly wrong.

The naive merge adds the scores. It never works: BM25 runs to tens or hundreds while cosine tops out at 1.0, so the keyword channel wins every query and the vector channel is decoration. Each list has to be put on a common scale first.

def normalize(scores):
    lo, hi = min(scores.values()), max(scores.values())
    if hi == lo:
        return {k: 0.0 for k in scores}   # one doc or all tied: no signal to pass on
    return {k: (v - lo) / (hi - lo) for k, v in scores.items()}

def fuse(bm25, vec, alpha=0.6):
    b, v = normalize(bm25), normalize(vec)
    pooled = {d: alpha * b.get(d, 0.0) + (1 - alpha) * v.get(d, 0.0)
              for d in set(bm25) | set(vec)}
    return sorted(pooled, key=pooled.get, reverse=True)

bm25 = {'d-sku': 14.2, 'd-policy': 3.1, 'd-faq': 0.9}
vec  = {'d-policy': 0.81, 'd-faq': 0.74, 'd-sku': 0.31}
print(fuse(bm25, vec))

Input: the two scored lists above, alpha=0.6 favoring keywords.

Output: ['d-sku', 'd-policy', 'd-faq'] — combined 0.600, 0.499, 0.344.

Read the numbers against the raw inputs. d-sku scored only 0.31 from the vector channel, but its BM25 score was the best in the list, so after normalization it holds 1.0 there and takes the top slot. d-policy is the vector winner at 0.81 and lands second because its keyword showing was weak. Min-max per query is what makes alpha mean anything: without it, alpha is a rounding error next to a BM25 of 14.2. Documents present in only one list get 0.0 in the other — strong single-channel hits can still surface, they just cannot pretend they were corroborated.

When score ranges are unknown or unstable, rank-based fusion removes the calibration problem entirely:

def rrf(rankings, k=60):
    totals = {}
    for ranking in rankings:
        for rank, doc in enumerate(ranking, start=1):   # ranks are 1-based
            totals[doc] = totals.get(doc, 0.0) + 1 / (k + rank)
    return sorted(totals, key=totals.get, reverse=True)

print(rrf([['d-sku', 'd-policy'], ['d-policy', 'd-faq', 'd-sku']]))

Output: ['d-policy', 'd-sku', 'd-faq'] — d-policy appears in both lists and wins on consistency even though d-sku holds the top keyword rank. RRF throws the scores away and uses positions, so it is immune to scale differences between channels; its cost is that score magnitude information is discarded. Fusing Multiple Retrievers covers the k constant and per-retriever logging.

Which channel deserves the weight depends on the query, and the query mix is a property of your product:

Failure modes that show up in real hybrid stacks:

• Fusing raw scores. The BM25 channel dominates every query and the demo still looks fine, because the demo queries were keyword-shaped.
• Fusing two top-8 lists. Fusion can only merge what each retriever returned. Over-fetch — top 40 or 50 per channel — fuse, then cut to the 8 that fit the prompt.
• Analyzer mismatch on identifiers. A standard analyzer splits SKU-7741-BLK into pieces, so neither channel matches the string the user pasted. Keyword or ngram subfields are the fix.
• Query embedded with a different model than the corpus. The vector channel returns near-random neighbors while BM25 still works, which hides the break — Embedding Models Explained covers the invariant.
• A weight tuned on paraphrase-shaped test queries. Those flatter vector search; tune alpha on sampled production queries or you will ship a keyword-proof system that fails on part numbers.
• Duplicate ids counted twice. The same document returned by both channels must be deduplicated before fusion, or double-listing reads as double evidence.

None of these can be settled by inspection. The weight is a number that moves recall@k, so it belongs on an evaluation set — labeled query-to-document pairs, recall@k and MRR computed per candidate alpha, best value shipped. That machinery, and how to know when the numbers lie, is Evaluating Embedding Quality.