RAG II — Retrieval, Hybrid Search, Reranking

Why this session exists

A retrieval system must find evidence before a generator can use it. A useful analogy is two librarians and an arbiter: one matches meaning, one matches words, and the arbiter reads a shortlist closely. None is infallible. Dense-only, sparse-only and hybrid are legitimate baselines; there is no universal industry stack, mandatory graph retriever, or guaranteed 10–30-point gain.

The learning goal is to design and diagnose a dense/lexical/fusion/reranking pipeline, including query transformations, permissions and evaluation. Prerequisites are chunking and index versioning from session 117, prompting from 116, and precision/recall. An exact ID such as ORA-01555 motivates lexical retrieval; a paraphrase such as “roll back a failed deployment” motivates semantic retrieval. Evaluate both query slices rather than assuming their failures are disjoint.

Dense, sparse and the candidate funnel

A bi-encoder computes document vectors offline and a query vector online. A normalized dot product is cosine; a model trained for another metric needs its own contract. A single vector compresses many possible document–query relationships. BM25 instead uses term frequency, rarity and document-length normalization. Tokenization still matters: lowercasing and splitting on spaces is not an identifier-aware analyzer. A rare term is not guaranteed to be decisive; spelling, punctuation, analyzer and IDF variant affect results.

One common positive-IDF BM25 variant is

Here repeated query terms are a declared implementation choice, not silently double-counted. BM25 implementations can use other IDF formulas, including negative values; “BM25 is always nonnegative” is not a general theorem. Dense and BM25 scores therefore cannot be added as if they shared units. Calibrated score fusion is possible; rank fusion avoids that calibration.

A candidate-only reranker cannot recover a missing document. For relevant set R and candidate set C, any selected subset S of C satisfies |S ∩ R| ≤ |C ∩ R|. This is a ceiling on the number of relevant documents, not proof that precision or every normalized metric decreases. Rewriting or a new retrieval call can expand the set, so the ceiling applies after the candidate pool is fixed. Multi-hop questions can have several missing passages rather than one uniquely failing stage.

Retrieve 50 per leg, rerank 20, select 5 is an experiment configuration, not a production standard. Increase the retrieval window when the evidence is absent; investigate ordering, context truncation, domain mismatch and duplicate chunks when it is present but not selected. Permission/version filters belong before evidence reaches the model. Filtering inside a capable index is useful, but exact search over an authorized subset or validated iterative retrieval are alternatives; postfiltering a fixed shortlist can starve results.

Reciprocal Rank Fusion worked example

For one-based ranks, absent documents contribute zero:

For dense [A,B,C] and sparse [B,D,A], with c=60, B receives 1/62+1/61 ≈ .032522, A receives 1/61+1/63 ≈ .032266, D receives 1/62 ≈ .016129, and C receives 1/63 ≈ .015873. Thus B, A, D, C is the order. A smaller c emphasizes the top ranks more strongly; a larger c makes overlap comparatively important. The method avoids fitting score scales, not all tuning: constant, list depth, number of retrievers, weights, duplicates and ties remain decisions. Count a document at most once per list; stable tie-breaking makes regression tests replayable. Elastic's RRF specification documents both the rank constant and window size.

Joint scoring, late interaction and query rewriting

A cross-encoder processes the query and document together. Token interactions can help with negation, exact conditions and contextual relevance; truncation and training-domain mismatch can still hurt. There is no guaranteed quality dominance. It must compute query-dependent pair representations at query time, although candidates may be batched: N pairs do not mean N separate API requests. Scoring every corpus document can be expensive; approximate indexes are not guaranteed O(log N) for all data. Two-stage retrieval is an engineering tradeoff, not a mathematical necessity for every corpus.

ColBERT keeps multiple document token vectors. Its late-interaction score sums, over query tokens, the maximum similarity to a document token. This retains more local matching detail than a pooled vector while allowing document-side preprocessing, at a larger index/storage and query-interaction cost. It does not imply a universal speed–quality ordering or establish any particular company's deployment. ColBERT, 2020 explains the decoupling.

Query transformations solve different problems:

• Standalone rewrite: resolve “what about the second one?” using authorized conversation context; retain IDs, dates and negations.
• Multi-query: retrieve several hypotheses for “my card doesn't work,” such as authorization failure and unsupported card type, then deduplicate/fuse. The fan-out and model call cost must earn improved evidence recall.
• HyDE: generate a hypothetical document, embed it, then retrieve actual documents. The generated text is a search aid, never evidence. Fabrication can move the vector in the wrong direction; retain the original-query baseline. Gao et al. describe the generation–encoding–retrieval chain.
• Step-back and decomposition: a broad background query is not identical to decomposing a multi-hop question into dependent subquestions. For “which policy applies to this purchase?”, first resolve purchase date/jurisdiction, then retrieve the corresponding policy. Keep the provenance chain.

Historically, BM25 predates neural retrieval; Sentence-BERT was published in 2019 and DPR in 2020, not the reversed dates in the original timeline. HyDE's preprint appeared in December 2022. These milestones explain lexical, representation and query-generation approaches; they do not support a story that hybrid became mandatory in 2024.

Hands-on: exact local retrieval contracts

This program implements a small BM25 variant, cosine ranking, RRF, bounded reranking and multi-relevant-document metrics. The vectors and reranker scores are fixtures, not embeddings or cross-encoder inference. The oracle deliberately knows relevance so that the ceiling and ranking behavior are inspectable, not to report a model-quality gain.

import math
from collections import Counter, defaultdict
 
DOCS = {
    "A": "oauth refresh token authorization",
    "B": "error E-4821 damaged shipment",
    "C": "split documents into chunks",
    "D": "refresh page browser cache",
}
 
def bm25(query, docs, k1=1.2, b=0.75):
    if not docs:
        return []
    counts = {i: Counter(t.lower().split()) for i, t in docs.items()}
    avg = sum(sum(c.values()) for c in counts.values()) / len(counts)
    if not avg:
        return sorted(docs)
    scores = dict.fromkeys(docs, 0.0)
    for term in set(query.lower().split()):
        df = sum(term in c for c in counts.values())
        idf = math.log1p((len(docs)-df+0.5)/(df+0.5))
        for i, c in counts.items():
            f = c[term]
            scores[i] += idf * f * (k1+1)/(f+k1*(1-b+b*sum(c.values())/avg))
    return sorted(scores, key=lambda i: (-scores[i], i))
 
def cosine(a, b):
    if len(a) != len(b) or not a:
        raise ValueError("shape")
    den = math.sqrt(sum(x*x for x in a)*sum(x*x for x in b))
    if not den:
        raise ValueError("zero vector")
    return sum(x*y for x, y in zip(a, b))/den
 
def rrf(*lists, c=60):
    if c <= 0:
        raise ValueError("positive rank constant required")
    scores = defaultdict(float)
    for ids in lists:
        for rank, i in enumerate(dict.fromkeys(ids), 1):
            scores[i] += 1/(c+rank)
    return sorted(scores, key=lambda i: (-scores[i], i)), dict(scores)
 
def rerank(ids, scorer, k):
    if k < 1:
        raise ValueError("positive k")
    unique = list(dict.fromkeys(ids))
    return sorted(unique, key=lambda i: (-scorer(i), i))[:k]
 
def metrics(ids, relevant, k):
    if not relevant or k < 1:
        raise ValueError("nonempty relevance set and positive k")
    ids = list(dict.fromkeys(ids))[:k]
    hit = len(set(ids) & relevant)
    return {"recall": hit/len(relevant), "precision": hit/k}
 
assert bm25("E-4821", DOCS)[0] == "B"
assert math.isclose(cosine([1, 1], [2, 2]), 1)
ids, scores = rrf(["A", "B", "C"], ["B", "D", "A"])
assert ids == ["B", "A", "D", "C"]
assert math.isclose(scores["A"], 1/61+1/63)
assert rrf(["A", "A", "B"])[0] == ["A", "B"]
relevant = {"A", "C"}
ranked = rerank(["D", "A"], lambda i: int(i in relevant), 2)
assert ranked == ["A", "D"]
assert metrics(ranked, relevant, 2) == {"recall": .5, "precision": .5}
assert "C" not in ranked  # a perfect ordering oracle cannot recover absence
assert rerank([], lambda i: 1, 2) == []
try:
    cosine([0, 0], [1, 2])
except ValueError:
    pass
else:
    raise AssertionError("zero-vector negative control")
# End-to-end fixture funnel: every leg returns document IDs, never mixed tuples.
vectors = {"A": [1.,0.], "B": [0.,1.], "C": [.8,.2], "D": [.6,.4]}
query_vector = [1.,0.]
dense = sorted(vectors, key=lambda i: (-cosine(query_vector, vectors[i]), i))
fused, _ = rrf(dense, bm25("oauth refresh", DOCS))
selected = rerank(fused[:3], lambda i: int(i == "A"), 2)
assert selected[0] == "A"
 
def multi_query(query, rewrite, retrieve, max_variants=3):
    variants = rewrite(query)
    if not isinstance(variants, list) or len(variants) > max_variants:
        raise ValueError("bounded rewrite list required")
    if any(not isinstance(v, str) or not v.strip() or len(v) > 300 for v in variants):
        raise ValueError("bounded nonempty query strings")
    queries = list(dict.fromkeys([query] + variants))  # retain original intent
    return rrf(*(retrieve(q) for q in queries))[0]
 
def hyde(query, generate_document, encode, retrieve_vector):
    hypothetical = generate_document(query)
    if not isinstance(hypothetical, str) or not 1 <= len(hypothetical) <= 1000:
        raise ValueError("bounded hypothetical document")
    return {"search_aid": hypothetical,
            "evidence_ids": retrieve_vector(encode(hypothetical))}
 
assert multi_query("card failure", lambda q: ["E-4821"],
                   lambda q: bm25(q, DOCS))[0] in DOCS
hyp = hyde("refresh?", lambda q: "invented refresh explanation", lambda s: [1.,0.],
           lambda v: sorted(vectors, key=lambda i: -cosine(v,vectors[i]))[:2])
assert set(hyp["evidence_ids"]) <= set(DOCS)
assert hyp["search_aid"] not in hyp["evidence_ids"]  # generated aid is not evidence
try:
    multi_query("q", lambda q: ["a"]*4, lambda q: [])
except ValueError:
    pass
else:
    raise AssertionError("rewrite fanout must be bounded")
print("retrieval contracts pass; no neural model evaluated")

From the local contract to Chroma and a cross-encoder

The original service recipe loaded all documents into an in-memory BM25 corpus and used PersistentClient, get_collection, query(query_embeddings=..., n_results=...) and CrossEncoder.predict(pairs). Preserve this integration as a design recipe, not a verified service run: use the source-reviewed Chroma 1.1.1 API contract, pin the embedding/reranker library and model revisions, read the collection's metric/embedding dimension, use the same embedding preprocessing at query and ingest, and explicitly request cosine if that is intended. A collection does not default to cosine merely because the code computes embeddings.

Build BM25 over the same authorized, versioned snapshot as the dense store. Estimate RAM from actual token counts and Python/object overhead rather than asserting “one million chunks fits.” Clamp k to available eligible documents, handle empty collections/pairs, batch cross-encoder pairs, and keep candidate IDs/provenance associated with scores. Do not infer tuple shape from fn.__name__: adapt every leg to a common list[document_id] interface before computing metrics. On updates/deletes, refresh both legs consistently. Add timeouts, per-tenant isolation, bounded retries and stale-version rejection before production use. The Sentence Transformers retrieve/rerank guide supports the two-stage mechanism, not this recipe's unexecuted end-to-end behavior.

Production diagnosis and practice solutions

Compare dense, sparse, fusion and reranking on the same held-out queries and relevance sets. Recall@k is hits divided by all relevant documents; precision@k is hits divided by k under the declared convention. With one gold document, recall@k reduces to hit rate, not a general multi-document metric. Also record MRR for first relevant rank and graded NDCG when relevance is ordinal. Stratify identifiers, paraphrases, ambiguous follow-ups, multi-hop and unauthorized queries; record candidate recall, final selection, latency and token cost separately.

The original answer-engine, enterprise and commodity-API “war stories” supplied no auditable incident evidence. Treat their mechanisms as three explicit scenarios: (1) lexical IDs fail, so test sparse alongside dense; (2) the gold chunk is at rank 8 outside a top-3 prompt, so test reranking; (3) an external reranker reduces hosting work but adds network/privacy/version constraints. No named-company migration, fixed 200ms cost, 15–30-point improvement, or 100K-label threshold is asserted.

Worked practice: label 20 or more representative questions with all acceptable source IDs. Preserve a holdout and repeat the same candidate selection with and without reranking. If recall@50 is low, inspect indexing, chunk context, authorization and query representation before the reranker. If it is high but recall@5 is low, a reranker is a candidate intervention, not an unambiguous proof of the cause. A larger embedding model may change query latency as well as require re-embedding; widening context can improve coverage or distract generation. Select the cheapest configuration meeting measured quality and latency constraints.

Recall and stretch: complete answers

1. Dense versus sparse: learned semantic matching can retrieve paraphrases; lexical matching can retain rare identifiers. Both can fail on the same query; analyzer and domain tests decide.
2. Why RRF without score calibration? Reciprocal one-based ranks share a scale. Constant, window and duplicate policy still need validation; consensus is not automatically truth.
3. Bi-encoder versus cross-encoder: independent preprocessing supports reusable document vectors; joint pair scoring allows richer interactions at extra query work, with no universal quality guarantee.
4. No reranker gain? Check missing candidates, domain mismatch, truncation, duplicate candidates, metric ceiling and bad labels. A reranker can also make ranking worse.
5. HyDE: encode a hypothetical document to search for real evidence; generated falsehoods can hurt retrieval and must not be cited.
6. Multi-query workload: ambiguous card-support wording can branch into authorization, expiry and compatibility hypotheses. Keep the original and measure fan-out benefit/cost.
7. Why not directly add scores? Cosine has a bounded geometric scale; BM25 varies with analyzer/corpus/variant. RRF or validated calibration avoids arbitrary scale dominance.
8. When add hybrid? When a held-out query slice gains relevant evidence at acceptable cost, not when an invented 10% or 80% threshold is crossed.
9. What funnel? 50→20→5 is an illustrative starting configuration; choose windows with candidate-recall, latency and context-budget measurements.
10. ColBERT stretch: sum query-token MaxSim values against separately encoded document token vectors. Compare its index size and quality to pooled and joint encoders; do not assume a fixed ranking among architectures.

Explain aloud where a missing chunk was lost, calculate the A/B RRF example, and justify one intervention from measured evidence rather than a promised gain. Retrieval feeds agents (120), evaluation (122) and multimodal search (125); it also needs incremental updates and cost monitoring, without mislabeling sessions 127/128 as streaming/cost lessons.

Optional resources and onward navigation

The original optional pre-read links are retained as navigation, not fact-check evidence or promises about video length, quality or API stability. Use the versioned primary sources in the teaching for consequential claims. Original pacing was a suggested 90-minute session, not a completion guarantee.

• RAG From Scratch: Part 1 (Overview)
• Building Production-Ready RAG Applications: Jerry Liu
• Cohere Rerank / cross-encoder rerankers tutorial
• Hybrid search 101 — dense + sparse
• Reciprocal Rank Fusion in Elasticsearch
• Precise Zero-Shot Dense Retrieval without Relevance Labels (HyDE)

Next in the original sequence: S119 · Vector Databases — pgvector, HNSW, IVF. Publication and release navigation are adjudicated by the parent.

Hub: The 6-Month Learning Plan. The original 130-session scaffold is retained as navigation, not evidence that every destination has been reviewed.

Review and execution boundary

Reviewed against complete original and pre-edit current teaching with claim-specific primary evidence and independently replayed coverage. The executable CPU program is a bounded teaching test, not a pretrained model, serving, RAG or training benchmark. External model/service recipes remain source-supported and unexecuted. No weights, datasets, GPU, paid API or new service were used. Source publication is not end-to-end application acceptance.