Search & Retrieval: Reading Guide
A vector index can be fast and accurate at finding its nearest neighbors while returning poor answers for the user. The embedding defines what “near” means; the index approximates that search; the surrounding product decides how candidates become useful results. Learn to separate those responsibilities before tuning an index.
This path has two chapters on ANN mechanics and benchmarking, followed by a seven-chapter evaluation series. Start with vectors, dot products and distances. If vector length and feature scaling are unfamiliar, use the normalization notes first.
Follow the query through three responsibilities
For a query such as “waterproof hiking boots,” an embedding may place hiking products nearby. Approximate search tries to find those neighbors efficiently. Product requirements still matter: a nearby result may be unavailable, fail the waterproof requirement or be ranked below a more useful alternative.
The model, preprocessing and distance define the neighborhood. Save their versions with the corpus.
Understand the distance convention →The index spends work to discover candidates. Compare them with exact neighbors under the same rules.
Inspect graph and partition search →Filtering, reranking and context construction affect what the user receives. Evaluate against the task.
Choose the appropriate evaluation →Keep two references distinct. Exact neighbors are defined by the vectors and scoring rule; they measure approximation error. Relevance judgments are defined by the user’s information need; they measure usefulness. A change can improve one comparison while leaving the other unchanged.
First understand the search, then measure its cost
Follow the hierarchy into a graph, distinguish pending exploration from retained candidates, and see how a partition boundary can hide a nearby item. Separate build parameters from query effort.
Construct exact references, draw the timing boundary, inspect tail observations and choose operating points that satisfy both quality and memory constraints.
The first chapter uses a fixed small graph and two illustrative partitions. Predict whether an unexplored route can contain a better neighbor, then step the figure forward. The second chapter uses constructed timing and operating-point examples to teach measurement choices. Neither presents those figures as results from a real deployment.
Match the comparison to what you changed
“Try another configuration” is easier to interpret when you know which artifacts must change with it. Keep the task and evaluation queries fixed wherever the comparison permits; record the differences explicitly.
| Change | Recompute or rebuild | Useful comparison |
|---|---|---|
| Query effort: ef or search_k | Run new queries on the existing index | Recall and latency against the same exact neighbors |
| Graph links, tree count or construction effort | Build the new index configuration | Build cost, memory and query tradeoffs with vectors fixed |
| Embedding model or preprocessing | Regenerate affected vectors, rebuild the index and compute matching exact references | Human relevance as well as approximation within each new geometry |
| Eligibility rule | Compute the appropriate eligible exact neighbors and rerun filtered search | Quality, result fill and latency under matching filters |
| Reranker | Score the same candidate IDs for an isolated ranking comparison | Judged ranking quality; separately check the deployed pipeline |
For example, increasing ef explores the graph more broadly; it does not alter the stored embeddings. Replacing the embedding model changes the search geometry itself. Comparing that new index with the old model’s exact neighbors would mix representation change with approximation error.
Continue from neighbor agreement to search quality
The evaluation reading guide connects seven focused chapters. Choose the next one from the evidence you are missing:
- Define relevance and build the test collection before treating a score as evidence of usefulness.
- Use binary ranking metrics or graded nDCG to express the retrieval question you care about.
- Locate stage losses, then compare matched queries with uncertainty.
- Validate user outcomes or audit evidence, answers and citations for the deployed experience.
A high ANN overlap score means the index agrees with its reference. It does not establish a relevance gain, a reliable improvement across queries or a successful user experiment. Each of those conclusions has a chapter and a corresponding measurement.
Build one small, auditable experiment before scaling up
Begin with a query and corpus small enough to inspect. Confirm the distance ordering, eligibility and exact result. Run approximate search, identify any missing neighbors, and connect the miss to the graph or partition behavior. Then broaden to representative queries and measure the intended service boundary.
Keep a run record with corpus and query versions, vector settings, item IDs, filter rules, index build parameters, query effort and software/hardware details. Save per-query results and latencies, including errors and empty results. This lets you revisit an aggregate when a configuration behaves unexpectedly.
For the code, start with the exact-reference example. For metric conventions, use the standard-library ranking lab. Read optional implementation details after the main example is clear, and change one factor at a time.
Connect indexing to the larger system
The RAG indexing overview places candidate search alongside document and chunk design. The evidence-coverage example follows retrieved material into the actual context sent to a generator.
Use the saved trace to decide whether the next change belongs in representation, candidate discovery, filtering, reranking or answer generation. Optimize a measured bottleneck and validate the outcome at the boundary it affects.
References and further reading
- Malkov and Yashunin: HNSW — the graph-search foundation expanded in the mechanics chapter.
- Spotify Annoy — construction, query effort and memory-mapped index usage.
- Faiss: metric types and distances — the score and normalization conventions that an exact reference must match.
- Manning, Raghavan and Schütze: ranked-retrieval evaluation — relevance-based metrics developed in the linked evaluation series.