Module 3: RAG Infrastructure & Retrieval · 2.75h

01 · UNDERSTAND

Day 16 Theory — Hybrid Search, Filtering and Reranking

Why one retrieval signal is rarely enough

Semantic search is excellent when wording differs but meaning is similar. Keyword search is excellent when exact identifiers, product names, error codes, or rare terms matter.

Support data contains both.

A query such as ORA-12514 may be served better by lexical matching than by semantic similarity. A query such as “login keeps rejecting my phone” may benefit from embeddings.

Hybrid search combines multiple signals.

Lexical and semantic retrieval

Lexical retrieval scores overlap in words or terms. BM25 is a common ranking approach.

Semantic retrieval compares embedding representations.

Hybrid systems retrieve or score with both, then combine rankings.

Score scales may not be comparable

A semantic similarity score and BM25 score can have completely different numeric ranges. Simply adding raw values can produce meaningless weighting.

Rank-based fusion methods avoid assuming scores share the same scale.

Reciprocal Rank Fusion (RRF), for example, combines positions from multiple ranked lists rather than raw scores.

Metadata filtering narrows the search space

Filters are not only performance optimizations. They can encode product, language, tenant, version, region, or permission boundaries.

Example:

query = "reset SSO"
filter = product:identity AND tenant:acme

Authorization-relevant filters must be derived from trusted application identity, not model-generated text.

Reranking adds a second-stage judge

Initial retrieval is optimized for finding candidates efficiently.

A reranker can examine the query and each candidate more deeply to reorder a smaller candidate set.

large corpus
  ↓ fast retrieval
20 candidates
  ↓ reranker
5 strongest candidates

Reranking can improve precision, but adds latency and cost. Measure whether the gain is worthwhile.

Diversity versus duplicates

Chunk overlap and near-duplicate documents can fill the top results with almost identical content.

A retrieval pipeline may need deduplication or diversity strategies so the model receives broader useful evidence.

Tune the whole pipeline with evals

Key knobs include:

lexical/semantic weighting,
candidate counts,
metadata filters,
reranker model,
final top-k.

Do not tune each knob independently by intuition. Compare against the Day 15 evaluation set and inspect failure slices.

Service Desk connection

Today the Service Desk becomes better at both natural-language problems and exact operational identifiers.

The principle is:

Use multiple retrieval signals for the strengths they provide, enforce trusted filters before generation, and spend reranking cost only where evaluation shows it improves useful evidence.

02 · APPLY

Lesson Overview

This is the applied companion for Day 16. Read DAY_16_THEORY.md first for the beginner-first teaching of Hybrid Search, Filtering and Reranking. Then use the real service-desk-day-16/ project to trace, run, debug, and explain the concept.

Service Desk Alignment

Day 16 adds Hybrid Search, Filtering and Reranking to the running Service Desk. Start with pipeline.py, filtering.py, vector_search.py, rrf.py, bm25.py, corpus.py, then follow imports and tests to identify the actual runtime path.

Why This Topic Matters

The theory chapter explains why Hybrid Search, Filtering and Reranking is needed. Here the goal is evidence: identify where the capability is implemented, what observable behavior changes, and how the repository proves both success and failure behavior.

System Architecture

This is a repository surface map, not a claim that modules call each other in the displayed order. The modules are ranked by relevance to today's theory.

graph LR
    T[Day 16: Hybrid Search, Filtering and Reranking]
    T --> M1[pipeline.py]
    T --> M2[filtering.py]
    T --> M3[vector_search.py]
    T --> M4[rrf.py]
    T --> M5[bm25.py]
    T --> M6[corpus.py]
    T --> M7[reranker.py]
    T --> M8[benchmark.py]

Follow imports and tests to discover the actual runtime flow.

Repository Implementation Map

Use the real Day 16 repository, not a fabricated sample, to connect theory to implementation.

Theory concepts to locate:

Why one retrieval signal is rarely enough
Lexical and semantic retrieval
Score scales may not be comparable
Metadata filtering narrows the search space

Most relevant implementation modules first:

service_desk/pipeline.py
service_desk/filtering.py
service_desk/vector_search.py
service_desk/rrf.py
service_desk/bm25.py
service_desk/corpus.py
service_desk/reranker.py
service_desk/benchmark.py
service_desk/query_rewriter.py
service_desk/models.py

Follow imports/calls from the relevant module and confirm behavior in tests. Record input → mechanism → observable output/state → failure evidence.

Code Walkthrough & Mechanics

Read pipeline.py, filtering.py, vector_search.py, rrf.py, bm25.py, corpus.py, reranker.py, benchmark.py, query_rewriter.py, models.py with these theory sections beside you:

Why one retrieval signal is rarely enough — locate its implementation and evidence.
Lexical and semantic retrieval — locate its implementation and evidence.
Score scales may not be comparable — locate its implementation and evidence.
Metadata filtering narrows the search space — locate its implementation and evidence.
Reranking adds a second-stage judge — locate its implementation and evidence.

For each concept identify the real function/class/protocol boundary, its input/state, its observable result, and the assertion in tests/test_hybrid_search.py that proves the behavior. If a concept has no implementation or evidence, record that as a gap rather than inventing one.

Common Mistakes & Debugging Guidance

Use the theory—not generic timeout or .env advice—to decide what can fail today.

Failure lens: revisit Metadata filtering narrows the search space, Reranking adds a second-stage judge.

Reproduce the smallest case that violates one of those expectations.
Trace the real Day 16 modules until you find the first incorrect state/output/decision.
Use tests/test_hybrid_search.py as executable evidence.
Add a regression test if the failure is not already represented.
Fix the smallest responsible boundary and rerun the relevant test before the full suite.

Your debugging explanation must name the topic-specific invariant that failed, not merely say “an exception occurred.”

Practical Lab Instructions

Summarize these theory ideas before opening the implementation:
- Why one retrieval signal is rarely enough
- Lexical and semantic retrieval
- Score scales may not be comparable
- Metadata filtering narrows the search space
Inspect the most relevant real Day 16 modules first:
- service_desk/pipeline.py
- service_desk/filtering.py
- service_desk/vector_search.py
- service_desk/rrf.py
- service_desk/bm25.py
- service_desk/corpus.py
- service_desk/reranker.py
- service_desk/benchmark.py
- service_desk/query_rewriter.py
- service_desk/models.py
Inspect the automated evidence:
- tests/test_hybrid_search.py

Establish the baseline:

cd service-desk-day-16
PYTHONPATH=. pytest tests/test_hybrid_search.py -q

Trace one theory concept through the actual nested modules and tests.
Run one success case and record input → mechanism → observable result.
Exercise one topic-specific failure/boundary case and name the invariant that protects the system.
Re-run the relevant tests and explain theory → implementation → evidence.

Done when: another student can reproduce your trace without relying on an invented sample.

Key Takeaways

Be able to explain Why one retrieval signal is rarely enough and point to its implementation/evidence in Day 16.
Be able to explain Lexical and semantic retrieval and point to its implementation/evidence in Day 16.
Be able to explain Score scales may not be comparable and point to its implementation/evidence in Day 16.

Knowledge Check & Scenario Questions

Concept: Using Why one retrieval signal is rarely enough, explain the engineering problem Day 16 is solving without naming a framework as the answer.
Mechanism: How does Lexical and semantic retrieval appear in the real project? Start from service_desk/pipeline.py and name the observable state/output/event that changes.
Failure: For Score scales may not be comparable, describe one incorrect implementation or boundary condition and the evidence you would expect in tests/test_hybrid_search.py.
Design review: Which assumption in today's design would you verify before reusing this implementation in a different production system?

Official References

Pinecone Hybrid Search & Sparse-Dense Vectors: https://www.pinecone.io/learn/hybrid-search-intro/
Reciprocal Rank Fusion (RRF) Research Paper: https://plg.uwaterloo.ca/~gvcormac/cormacksigir09-rrf.pdf

03 · EXPLAIN

Interview checkpoint

Explain one design decision from this lesson, the alternative you rejected, and the failure mode or evidence that justified your choice.