0Pricing
AI Agents · Lesson

Re-ranking with Cross-Encoders

Retrieve top-50 with cheap embeddings, then re-rank top-5 with a slower cross-encoder for higher precision.

Re-ranking with Cross-Encoders is a free AI Agents lesson on CoddyKit — lesson 1 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Agents learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

Why Re-Rank?

Embedding similarity is a fast, coarse first pass. It often pulls in chunks that share KEYWORDS with the query but are not actually relevant to the QUESTION.

A re-ranker takes (query, chunk) pairs and computes a much more precise relevance score.

Bi-Encoder vs Cross-Encoder

Two architectures for text similarity:

  • Bi-encoder — embeds query and chunk separately, computes cosine. Fast (vectorise once, reuse), less accurate.
  • Cross-encoder — runs (query, chunk) jointly through the model. Slow (per-pair), much more accurate.

Two-Stage Retrieval

The pattern that combines them:

  1. Bi-encoder retrieves top-50 candidates fast
  2. Cross-encoder re-ranks to top-5
  3. Top-5 go into the LLM prompt

You get bi-encoder speed + cross-encoder precision.

Using Cohere Rerank

The easiest production cross-encoder:

import cohere
co = cohere.Client(COHERE_KEY)

results = co.rerank(
    model='rerank-multilingual-v3.0',
    query='What is the refund policy?',
    documents=top_50_chunks,
    top_n=5
)
for r in results.results:
    print(r.index, r.relevance_score, top_50_chunks[r.index])

Open-Source Rerankers

BGE Reranker is the leading OSS option:

from sentence_transformers import CrossEncoder
model = CrossEncoder('BAAI/bge-reranker-base')

pairs = [(query, chunk) for chunk in candidates]
scores = model.predict(pairs)
ranked = [c for _, c in sorted(zip(scores, candidates), reverse=True)][:5]

Voyage Rerank

import voyageai
vo = voyageai.Client(api_key=VOYAGE_KEY)
res = vo.rerank(
    query=query,
    documents=candidates,
    model='rerank-2',
    top_k=5
)

When Re-Ranking Helps

  • Long candidate lists (50-200)
  • Subtle relevance differences
  • Queries that include negation or comparison
  • Multilingual cases

When It Does Not

  • Already-narrow candidate sets (top-5 from bi-encoder)
  • Latency-critical, low-stakes queries

Cost Trade-off

Cross-encoders run an inference per pair. Reranking 50 candidates is 50 model calls (cheap with Cohere/Voyage, slower if self-hosted).

Budget 50-200 candidates for rerank, more is rarely worth it.

Integrating into LangChain

from langchain.retrievers import ContextualCompressionRetriever
from langchain.retrievers.document_compressors import CohereRerank

compressor = CohereRerank(top_n=5)
retriever = ContextualCompressionRetriever(
    base_compressor=compressor,
    base_retriever=vector_retriever
)

Integrating into LlamaIndex

from llama_index.postprocessor.cohere_rerank import CohereRerank

rerank = CohereRerank(top_n=5)
query_engine = index.as_query_engine(
    similarity_top_k=20,
    node_postprocessors=[rerank]
)

Diversity-Aware Reranking

Sometimes you want diverse results, not just the top-scoring ones (which may be near-duplicates). MMR (Maximal Marginal Relevance) balances score with diversity:

from langchain.vectorstores import Chroma
results = store.max_marginal_relevance_search(query, k=5, fetch_k=20, lambda_mult=0.5)

Two-Stage Pattern

Why use bi-encoder + cross-encoder instead of just one?

Recap

Bi-encoder retrieves 50, cross-encoder reranks to 5. Use Cohere or Voyage in prod; BGE OSS if self-hosting. Significant accuracy improvement for moderate added cost.

Frequently asked questions

Is the “Re-ranking with Cross-Encoders” lesson free?

Yes — the full text of “Re-ranking with Cross-Encoders” is free to read here on the web, and the AI Agents course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Agents course, upgrade to CoddyKit PRO.

What will I learn in “Re-ranking with Cross-Encoders”?

Retrieve top-50 with cheap embeddings, then re-rank top-5 with a slower cross-encoder for higher precision. You practise AI Agents with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start AI Agents?

No prior experience is required. AI Agents on CoddyKit is structured for beginners through advanced learners; this is — lesson 1 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “Re-ranking with Cross-Encoders” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this AI Agents lesson?

Yes. Every AI Agents lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

  1. Re-ranking with Cross-Encoders
  2. HyDE: Hypothetical Document Embeddings
  3. Multi-Vector Retrieval (ColBERT)
  4. RAG Evaluation (RAGAS, Recall@K)
← Back to AI Agents