Parent-Child and Small-to-Big Retrieval
Store small child chunks for precise retrieval but return their larger parent chunks to the LLM for richer context, balancing retrieval precision with generation quality.
Parent-Child and Small-to-Big Retrieval is a free AI Engineering Academy lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Engineering Academy learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
The Precision vs Context Dilemma
RAG systems face a tension: small chunks are retrieved with high precision because each chunk is focused on one idea, but they lack the surrounding context the LLM needs to generate a complete answer. Large chunks provide rich context but reduce retrieval precision because they match many queries weakly instead of one query strongly. Parent-child chunking solves this dilemma.
The Parent-Child Architecture
In parent-child chunking, you create two layers of chunks from the same document. Child chunks are small (e.g., 1-3 sentences) and are embedded and indexed for retrieval. Parent chunks are larger sections (e.g., entire paragraphs or pages) that are stored separately. When a child is retrieved, you return its parent to the LLM instead.
Building the Chunk Hierarchy
The first step is to split the document into large parent chunks, then split each parent into smaller child chunks. Each child chunk keeps a reference — typically a parent_id metadata field — pointing back to its parent. This mapping allows you to look up the full parent passage given any retrieved child.
from langchain.text_splitter import RecursiveCharacterTextSplitter
parent_splitter = RecursiveCharacterTextSplitter(chunk_size=1500, chunk_overlap=0)
child_splitter = RecursiveCharacterTextSplitter(chunk_size=300, chunk_overlap=30)
parent_chunks = parent_splitter.split_documents(docs)
child_chunks = []
for i, parent in enumerate(parent_chunks):
children = child_splitter.split_documents([parent])
for child in children:
child.metadata['parent_id'] = i
child_chunks.extend(children)Indexing Only Child Chunks
Only the child chunks are embedded and stored in the vector database. The parent chunks are stored in a separate key-value store (an in-memory dictionary, Redis, or a document database). This keeps the vector index dense and precise while the rich context lives outside it.
# Store parents in a docstore
parent_store = {i: chunk.page_content for i, chunk in enumerate(parent_chunks)}
# Embed and index only children
vectorstore = Chroma.from_documents(
child_chunks,
embedding=OpenAIEmbeddings()
)Retrieval: Child In, Parent Out
During retrieval, the user query is embedded and matched against child chunks. The top-k child chunks are found, and their parent_id references are resolved by looking up the parent store. The parent passages — not the children — are then injected into the LLM prompt. The LLM receives broad context; retrieval was precise.
def retrieve_with_parents(query, vectorstore, parent_store, k=5):
child_results = vectorstore.similarity_search(query, k=k)
seen_parent_ids = set()
parent_contexts = []
for child in child_results:
pid = child.metadata['parent_id']
if pid not in seen_parent_ids:
parent_contexts.append(parent_store[pid])
seen_parent_ids.add(pid)
return parent_contextsLangChain ParentDocumentRetriever
LangChain provides the ParentDocumentRetriever class that implements this pattern out of the box. You provide a parent splitter, child splitter, a vectorstore for child embeddings, and a docstore for parent documents. It wires up the hierarchy and handles retrieval transparently.
from langchain.retrievers import ParentDocumentRetriever
from langchain.storage import InMemoryStore
store = InMemoryStore()
retriever = ParentDocumentRetriever(
vectorstore=vectorstore,
docstore=store,
child_splitter=child_splitter,
parent_splitter=parent_splitter
)
retriever.add_documents(docs)
results = retriever.invoke('What is the refund policy?')Small-to-Big Retrieval Explained
Small-to-big retrieval is another name for the same concept: you retrieve small, precise chunks but then expand them to their surrounding context before sending to the LLM. Some implementations expand not to a fixed parent but to a window of adjacent sentences — giving the model the sentences before and after the matched chunk for contextual continuity.
def retrieve_with_window(query, vectorstore, sentences, window=2, k=5):
results = vectorstore.similarity_search(query, k=k)
expanded = []
for r in results:
idx = r.metadata['sentence_index']
start = max(0, idx - window)
end = min(len(sentences), idx + window + 1)
expanded.append(' '.join(sentences[start:end]))
return expandedDeduplicating Parent Chunks
Multiple child chunks from the same parent may all be retrieved for one query. Without deduplication, the same parent passage would appear multiple times in the prompt, wasting tokens. Always deduplicate by parent ID before assembling the context. The code example in the retrieve function above handles this with a seen_parent_ids set.
When to Use Parent-Child Chunking
Parent-child retrieval works best when your documents have clear hierarchical structure: chapters with sections, articles with paragraphs, or wikis with subsections. It is particularly effective for long technical documentation where precise questions need localized answers but those answers only make sense within a broader section of context.
Choosing Child and Parent Sizes
A typical configuration is: child chunks of 200-400 tokens (focused single ideas) and parent chunks of 1000-2000 tokens (complete sections). If child chunks are too small, they become individual sentences that lack meaning on their own. If parent chunks are too large, you start reintroducing the context dilution problem you were trying to avoid.
Comparing Approaches: A Summary
To summarize the chunking strategies so far: fixed-size is fast but breaks context; semantic chunking preserves topic coherence; parent-child optimizes both retrieval precision and LLM context richness. For most production RAG systems handling long documents, parent-child chunking delivers the best retrieval quality at manageable complexity.
Quick Check
Test your understanding of parent-child chunking from this lesson.
Lesson Recap
In this lesson you learned: child chunks provide precise retrieval while parent chunks provide rich context, LangChain's ParentDocumentRetriever implements this automatically, and deduplication by parent ID prevents context repetition. Next up we explore document-specific chunking strategies for code files and HTML documents.
Frequently asked questions
Is the “Parent-Child and Small-to-Big Retrieval” lesson free?
Yes — the full text of “Parent-Child and Small-to-Big Retrieval” is free to read here on the web, and the AI Engineering Academy course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Engineering Academy course, upgrade to CoddyKit PRO.
What will I learn in “Parent-Child and Small-to-Big Retrieval”?
Store small child chunks for precise retrieval but return their larger parent chunks to the LLM for richer context, balancing retrieval precision with generation quality. You practise AI Engineering Academy with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start AI Engineering Academy?
No prior experience is required. AI Engineering Academy on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Parent-Child and Small-to-Big Retrieval” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this AI Engineering Academy lesson?
Yes. Every AI Engineering Academy lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Why Naive Chunking Hurts Retrieval
- Semantic Chunking with Embedding Similarity
- Parent-Child and Small-to-Big Retrieval
- Document-Specific Strategies for Code and HTML