Vector databases & ANN at scale
P11.ir-rag.03 · Audience: guest, it-ml, language-pro · Prerequisites: Information retrieval & hybrid search
Dense retrieval finds a query's nearest neighbours in vector space. Exact search compares against every stored vector — fine for the demo corpus, ruinous at 100M vectors. This module is the four pressures that forces on you, and the knobs (nprobe, quantization, sharding) that trade around the recall / latency / memory triangle.
Exact (flat) search compares the query to every stored vector: 1M × 768-d float32 ≈ 3 GB and a full scan per query; 100M ≈ 300 GB — no longer one machine's RAM, and the scan blows the latency budget. Too big to hold and too slow to scan.
| Pressure | Exact / flat | What has to give |
|---|---|---|
| Latency | O(N) scan/query | approximate index (HNSW / IVF) |
| Memory | full float32 vectors | quantization (PQ / scalar) |
| One machine | 300 GB won't fit | sharding across nodes |
| Throughput / HA | single copy | replication |
🎯 The headline trade is recall vs latency vs memory — every knob moves you around that triangle.
⚡ Interview Ref — the quick-scan Reference face
- Why not exact: O(N) scan + full float32 vectors → too slow and too big past ~1M–100M. Trade lives on the recall / latency / memory triangle.
- IVF: k-means cells (coarse quantizer) +
nprobecells scanned.nprobe= recall/latency knob;nprobe = cells⇒ exact. HNSW'sefSearchis the same trade. - HNSW vs IVF-PQ: graph walk (fast, RAM-heavy) vs cells + PQ codes (compact, RAM-bound scale).
- Quantization: PQ (m sub-vectors → codebook ids, 8–32×, lookup-table distances,
refinere-scores the shortlist); scalar int8 (4×, almost lossless). - Scale-out: shard (fan-out + merge), replicate (throughput + HA), blue/green rebuilds, hot-delta index for freshness.
Ask the mentor about this module
Ask a question about this content. The mentor explains and grounds its answer in what you are studying; asking is recorded as a learning signal, not a grade.
Try it yourself
A scratch console for this page's ideas — ungraded, nothing you run here is recorded.
Scratch console
A scratch console with the scientific stack (pandas, numpy, scikit-learn). Runs on the server — no network, resource-limited and measured.
Output appears here.
Where next?
Later in Information Retrieval & RAG