Metadata Filtering in Retrieval

Two enterprise customers share one vector collection: Acme and Northwind. A support agent at Acme searched "how are renewal discounts calculated" and the top result was Northwind's contract terms — the same vocabulary, a different company, and a clause the agent was never authorized to read.

Nothing about the search was broken. Cosine similarity did its job: the two documents genuinely are close. What was missing is the one thing similarity has no opinion about — who is asking. A shared index needs a predicate that says which rows this request may see, and that predicate has to run before ranking, not after.

request body                         server
{ "query": "...",                    session -> tenant_id = "acme"     trusted
  "tenant": "northwind" }   --->     embed the query
                                          |
                                          v
                              WHERE tenant_id = $session.tenant_id     built here, never read
                                          |                            from the payload
                                          v
                             distance ordering over surviving rows
                                          |
                                          v
                                    top-k passages

The bug in the incident was one line of plumbing: the filter value came from the request body, and a newer client that had no reason to send it sent nothing.

rows = search(vec, tenant=payload.get("tenant"))   # client picks the boundary; omit it and there is none
rows = search(vec, tenant=session.tenant_id)       # server derives it; there is no omitting it

Access control is not the only filter worth pushing down, but it is the only one where a mistake is a data breach rather than a bad answer. The rest — topic, date, product line — are precision filters that make results better, and they belong in the same WHERE clause for the same reason: a predicate that runs after ranking cannot guarantee the top-k it returns.

Post-filtering looks correct in code review and behaves badly under load: fetch the 50 nearest rows, then drop the ones that fail the filter. If ten of them belong to another topic, the caller receives 40 instead of 50 — and if the matching passage sits at global rank 60, it was never fetched at all. The filter belongs in the same statement as the ordering:

SELECT id, heading, 1 - (embedding <=> $1) AS score
FROM chunks
WHERE tenant_id  = $2
  AND topic      = ANY($3)
  AND updated_at >= $4
ORDER BY embedding <=> $1
LIMIT 8;

The WHERE narrows the candidate set before distance ordering, so the eight rows that come back are the eight nearest rows among the eligible ones — a guarantee the fetch-then-drop pattern cannot make.

async def search(vec, session, topics, since, k=8):
    return await pool.fetch(
        """SELECT id, heading, 1 - (embedding <=> $1) AS score
           FROM chunks
           WHERE tenant_id = $2 AND topic = ANY($3) AND updated_at >= $4
           ORDER BY embedding <=> $1
           LIMIT $5""",
        vec, session.tenant_id, topics, since, k,      # tenant_id from the session, never the payload
    )

Input: query renewal discount terms, session tenant acme, topics ["billing"], since 2026-06-01.

Output: three rows, all tenant_id = acme, best score first — ("ch_913", "Billing > Renewal discounts", 0.81), ("ch_447", "Billing > Tier thresholds", 0.74), ("ch_118", "Billing > Proration", 0.70).

How much the predicate helps depends on how much it removes, on an illustrative 1.2M-chunk corpus:

The filter columns need indexes or the shrunken candidate set still costs a scan; an unindexed tenant_id quietly turns the optimized query back into a full-collection search.

The row that has zero eligible documents is the case teams skip. A user who combines a narrow topic with a stale date window gets an empty result set, concludes the assistant is broken, and files a ticket — while the log records a perfectly ordinary search.

Treat constraints as an ordered ladder: hard constraints hold everything, soft constraints drop in a declared order when the result count falls below a floor.

def search_with_fallback(vec, tenant, topic, since, k=8):
    rows = run(vec, tenant=tenant, topic=topic, since=since, k=k)
    if rows:
        return rows, "filtered"
    rows = run(vec, tenant=tenant, topic=topic, since=None, k=k)   # 1. widen the time window
    if rows:
        return rows, "date-widened"
    rows = run(vec, tenant=tenant, topic=None, since=None, k=k)    # 2. drop the topic tag
    if rows:
        return rows, "topic-dropped"
    return [], "no-eligible-docs"                                  # the tenant filter is never relaxed

Input: tenant acme, topic billing, since 2026-09-01, where the strict pass returns no rows.

Output: rows from the tenant, ordered by distance, with the tag date-widened — and if widening is still empty, topic-dropped. Only when the tenant itself has nothing does the search come back empty.

The order matters more than the code. Drop the softest constraint first (topic), then the next (date); stop the moment results appear, because every dropped constraint is precision you gave back. The hard constraint is not on the ladder at all: the correct answer to "this tenant has no such document" is an empty list, never another tenant's passage dressed up as a result.

Failure modes to check before shipping:

• Client-supplied tenancy — the filter is a security boundary; it belongs to the session, not the payload.
• Post-filtering wearing a pre-filter's name — LIMIT applied before WHERE in the query plan returns a truncated, then discarded, window.
• A fallback that relaxes access control — the ladder exists to trade precision for recall, never to trade a boundary for a hit rate.
• Unlogged zero-row searches — count them and alert on the trend; a rising count means the default filters are too narrow, not that users are asking odd questions.