Metadata Filtering for Hybrid Search
Combine vector similarity with structured filters (date, author, tag) to get precise, contextual retrieval.
Metadata Filtering for Hybrid Search is a free AI Agents lesson on CoddyKit — lesson 2 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Agents learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Why Filtering?
Pure vector search returns the K most similar chunks — even if they belong to the wrong tenant, wrong language, or wrong document.
Metadata filters scope the search to relevant subsets, giving you precision AND recall.
Storing Metadata
Attach a dict of metadata to every chunk on ingestion:
metadata = {
'tenant_id': 'acme',
'language': 'en',
'source': 'help-docs',
'updated_at': '2024-08-12',
'tags': ['shipping', 'returns']
}
for k, v in metadata.items():
print(f"{k}: {v}")
Filtering in Pinecone
results = index.query(
vector=query_vec,
top_k=5,
filter={
'tenant_id': 'acme',
'language': 'en'
}
)Range Filters
Most vector DBs support range filters too:
filter = {
'updated_at': {'$gte': '2024-01-01'},
'score': {'$gt': 0.5}
}
print(filter)
In and Not-In
filter = {
'tags': {'$in': ['shipping', 'returns']},
'archived': {'$ne': True}
}
print(filter)
Filtering in Qdrant
Qdrant has a rich filtering language:
from qdrant_client.models import Filter, FieldCondition, MatchValue
filter = Filter(must=[
FieldCondition(key='tenant_id', match=MatchValue(value='acme')),
FieldCondition(key='language', match=MatchValue(value='en'))
])
results = client.search(
collection_name='docs',
query_vector=query_vec,
query_filter=filter,
limit=5
)Pre-Filter vs Post-Filter
Two strategies:
- Pre-filter — apply filter THEN search (correct, may be slow without indexed metadata)
- Post-filter — search then drop non-matching (fast but may return zero results)
Production systems do pre-filter with proper metadata indexes.
Hybrid Vector + Keyword
Combine semantic search with classical BM25 keyword search. Run both, combine scores (reciprocal rank fusion is the standard):
def rrf(vec_results, bm25_results, k=60):
scores = defaultdict(float)
for rank, doc in enumerate(vec_results):
scores[doc.id] += 1 / (k + rank)
for rank, doc in enumerate(bm25_results):
scores[doc.id] += 1 / (k + rank)
return sorted(scores.items(), key=lambda x: -x[1])Multi-Tenancy
In SaaS, EVERY query must filter by tenant_id. Hard-code it in your retrieval wrapper so it cannot be forgotten:
def search(query, tenant_id, top_k=5):
return index.query(
vector=embed(query),
top_k=top_k,
filter={'tenant_id': tenant_id}
)Access Control via Metadata
Store user/role permissions in metadata:
metadata = {
'visibility': 'public', # or 'internal', 'restricted'
'department': 'engineering',
'allowed_roles': ['engineer', 'manager']
}
# At query time:
filter = {'allowed_roles': {'$contains': current_user_role}}Recency Boost
To prefer newer documents, retrieve more candidates then re-score by recency:
candidates = index.query(vector=query_vec, top_k=50)
scored = [
(c.score * recency_factor(c.metadata['updated_at']), c)
for c in candidates
]
scored.sort(reverse=True)
final = scored[:5]Watch Metadata Cardinality
High-cardinality metadata (millions of unique values) makes indexes huge. Pre-aggregate when possible — e.g. store year-month instead of full timestamp for time filtering.
Always-On Filter
What filter should every multi-tenant agent ALWAYS include?
Recap
Metadata filters scope your search. Combine with hybrid (vector + BM25) for the best results. Multi-tenancy and access control rely on this.
Frequently asked questions
Is the “Metadata Filtering for Hybrid Search” lesson free?
Yes — the full text of “Metadata Filtering for Hybrid Search” is free to read here on the web, and the AI Agents course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Agents course, upgrade to CoddyKit PRO.
What will I learn in “Metadata Filtering for Hybrid Search”?
Combine vector similarity with structured filters (date, author, tag) to get precise, contextual retrieval. You practise AI Agents with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start AI Agents?
No prior experience is required. AI Agents on CoddyKit is structured for beginners through advanced learners; this is — lesson 2 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Metadata Filtering for Hybrid Search” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this AI Agents lesson?
Yes. Every AI Agents lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Pinecone, Weaviate, Qdrant: Comparison
- Metadata Filtering for Hybrid Search
- Updating and Deleting Vectors
- Choosing Distance Metrics (cosine, L2, dot)