Information Retrieval & RAG

Ask a production chat model about your company's refund policy and it answers fluently, in confident corporate register — and wrong, because the policy lives in a PDF that legal updated last Tuesday, and the model's weights were frozen months before that. No amount of prompting fixes this: the answer simply is not in the parameters. The fix is architectural — make the model look things up before it speaks. This track is about the looking, which turns out to be a deeper engineering problem than the speaking.

Retrieval is a funnel, not a lookup. The track opens by building the funnel's stages by hand — candidate generation over an inverted index, BM25 next to embedding search, and the fusion step that merges the two — because confusing these stages is the classic failure in both interviews and production. Then it insists you measure: re-ranking with cross-encoders, and the IR metrics (nDCG, MRR, recall@k) that tell you whether a change actually helped. Most RAG failures are retrieval failures, and without these numbers you are debugging the wrong component.

The second half scales and grounds. Vector databases and approximate nearest-neighbour indexes (HNSW and friends) buy speed by trading exact recall — you will see exactly what that trade costs and when it is safe. The capstone assembles retrieval-augmented generation properly: retrieved passages feeding the model, answers that cite their sources, and — the part naive RAG demos skip — a system that can abstain when the evidence is not there. Everything in P11's agents track stands on this foundation: an agent that cannot retrieve reliably only automates being wrong.