2단계 검색이 작동하는 이유

Every retrieval system faces a fundamental trade-off: recall measures how many relevant documents you find (did you miss any?), while precision measures how accurate the top results are (how many retrieved docs are actually relevant?). Maximizing both simultaneously is computationally expensive. Fast retrievers sacrifice precision for recall; precise rankers sacrifice speed for accuracy.

The two types of models at the heart of two-stage retrieval differ in how they see the query and document. A bi-encoder encodes the query and each document independently and measures similarity between their vectors — fast but limited by independent encoding. A cross-encoder sees the query and document concatenated as a single input, enabling deep interaction between them — highly accurate but O(n) complexity over the candidate set.

The first stage is a fast retriever — typically a bi-encoder with an approximate nearest neighbor index or a BM25 index — that retrieves a large candidate set (50-200 documents) with high recall but modest precision. The goal is not to be accurate; it is to not miss relevant documents. We cast a wide net and accept some false positives, knowing the second stage will clean them up.

The second stage takes the candidate set from stage 1 and re-scores each (query, document) pair using a cross-encoder that reads both together. Because it processes only 50-200 candidates (not the full corpus), it can afford the expensive joint encoding. The cross-encoder's deep attention over the concatenated input makes it far more accurate at estimating true relevance than a bi-encoder.

The two-stage design exploits a key asymmetry: the first stage's fast ANN search scales to millions of documents in milliseconds, while the second stage's accurate cross-encoder operates only on the small candidate pool. You get the scalability of approximate search with the accuracy of exact joint scoring. The overall pipeline is both fast and highly accurate — something neither stage achieves alone.

In a typical two-stage pipeline: stage 1 (vector ANN search over 1M documents) takes 5-20ms; stage 2 (cross-encoder over 100 candidates) takes 100-500ms depending on document length and hardware. The total latency budget is 150-600ms — acceptable for most applications. GPU acceleration in stage 2 can reduce re-ranking to under 30ms for short documents, making the pipeline competitive with single-stage retrieval on latency-sensitive applications.

The first stage candidate set size is a critical hyperparameter. Too small (say, 10) and relevant documents may be missed before re-ranking even starts. Too large (say, 500) and stage 2 latency blows up. The recall at N curve — how many relevant documents are captured at different values of N — guides this choice. Typical sweet spots are between 50 and 150 candidates, where recall is near-saturation but latency remains manageable.

The most powerful two-stage configuration pairs a hybrid retriever (dense + BM25) as the first stage with a cross-encoder as the second stage. Hybrid retrieval maximizes first-stage recall by combining semantic and keyword matching, and the cross-encoder then accurately selects the most relevant documents from the combined candidate pool. This configuration consistently achieves state-of-the-art retrieval quality on benchmarks.

If you want cross-encoder accuracy without managing your own model, both Cohere Rerank and Jina AI Reranker offer cloud-hosted re-ranking APIs. You send a query and a list of document texts, and receive relevance scores back. These APIs use large cross-encoder models (often 500M+ parameters) that outperform self-hosted small cross-encoders, at the cost of additional API latency (50-300ms) and pricing per reranked document.

Two-stage retrieval adds complexity and latency compared to single-stage. It is not always necessary. For small corpora under 10,000 documents, a single cross-encoder over the full set may be fast enough. For applications where latency under 100ms is critical and accuracy gains are modest, single-stage dense retrieval may be preferable. Use two-stage when you have a large corpus, high accuracy requirements, and can afford 200-500ms retrieval latency.

For corpora of tens of millions of documents, a three-stage pipeline is sometimes used: first stage retrieves 10,000 candidates with ANN, second stage re-ranks to 100 using a fast small cross-encoder, and third stage re-ranks to 5 using a large powerful cross-encoder. Each stage applies a more expensive and accurate model to a smaller set. This architecture is used by large-scale search engines and document Q&A systems.

Test your understanding of why two-stage retrieval works from this lesson.

In this lesson you learned: bi-encoders are fast but limited to independent query-document encoding, cross-encoders are accurate through joint encoding but too slow for full-corpus search, and two-stage retrieval combines both: a fast first stage for high recall followed by an accurate second stage for high precision. The first stage retrieves many more candidates than needed to avoid missing relevant documents. Next up we implement cross-encoder re-ranking with Cohere and BGE.