AI Agents · Lesson

Multi-Vector Retrieval (ColBERT)

Index multiple vectors per document (one per token or per chunk) for fine-grained matching.

Multi-Vector Retrieval (ColBERT) is a free AI Agents lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Agents learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

One Vector Is Not Enough

Standard RAG embeds each chunk into a single vector. That vector averages everything in the chunk — losing fine-grained signal.

Multi-vector retrieval stores MULTIPLE vectors per document and matches them more precisely.

ColBERT Idea

ColBERT (Khattab & Zaharia, 2020) embeds each TOKEN of the document and each token of the query, then computes max-similarity:

score(q, d) = sum over q_token in q: max over d_token in d: cos(q_token, d_token)

Why ColBERT Beats Single-Vector

Captures multiple aspects of a document (one vector cannot)
Handles long documents better
Higher recall on rare terms

Cost of ColBERT

Storing one vector per token instead of per chunk:

If a chunk has 500 tokens, that's 500x more storage
Each search compares many more pairs

Quantization and pruning reduce this 10-50x, but it's still costly.

ColBERTv2

ColBERTv2 adds residual compression — tokens cluster into centroids, and each token is stored as (centroid_id, small_residual). Storage drops 50x.

Running ColBERT

The RAGatouille library makes it easy:

# pip install ragatouille
from ragatouille import RAGPretrainedModel

rag = RAGPretrainedModel.from_pretrained('colbert-ir/colbertv2.0')
rag.index(collection=documents, index_name='my_corpus')

results = rag.search(query='What is the refund policy?', k=5)

Multi-Vector in Practice

Beyond ColBERT, other multi-vector approaches:

Parent-child — embed small chunks, retrieve large parents
Summary + chunks — embed both a doc summary and individual chunks
Hypothetical questions — embed Q&A pairs synthesized from each doc

Hypothetical Questions Pattern

def index_with_hypos(doc):
    # Generate 5 plausible questions this doc could answer
    questions = llm.invoke(f'Write 5 questions answered by this doc:\n{doc}').content.split('\n')
    for q in questions:
        vector_db.add(embed(q), payload={'doc': doc, 'question': q})
# Now queries that match a stored hypothetical question retrieve the doc.

Parent-Child with LangChain

from langchain.retrievers import ParentDocumentRetriever
from langchain.storage import InMemoryStore

store = InMemoryStore()
retriever = ParentDocumentRetriever(
    vectorstore=child_vectorstore,
    docstore=store,
    child_splitter=child_splitter,    # small
    parent_splitter=parent_splitter   # large
)
retriever.add_documents(documents)
# Indexes small chunks, returns large parents.

When to Use ColBERT

High-stakes search (legal, medical)
Long documents
When you can afford the storage cost

When to Skip

Small corpora
Latency-sensitive paths
Cost-constrained projects

Hybrid With BM25

For maximum recall, combine multi-vector retrieval with classical BM25 keyword search. Use Reciprocal Rank Fusion to merge.

ColBERT Trade-off

What is the main trade-off of using ColBERT-style multi-vector retrieval?

Recap

One vector per chunk loses information. ColBERT stores per-token vectors for higher precision. RAGatouille makes it easy. Use when recall matters and cost permits.

Frequently asked questions

Is the “Multi-Vector Retrieval (ColBERT)” lesson free?

Yes — the full text of “Multi-Vector Retrieval (ColBERT)” is free to read here on the web, and the AI Agents course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Agents course, upgrade to CoddyKit PRO.

What will I learn in “Multi-Vector Retrieval (ColBERT)”?

Index multiple vectors per document (one per token or per chunk) for fine-grained matching. You practise AI Agents with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start AI Agents?

No prior experience is required. AI Agents on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “Multi-Vector Retrieval (ColBERT)” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this AI Agents lesson?

Yes. Every AI Agents lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

← Back to AI Agents