Retrieval-Augmented Generation (RAG) Basics
Learn how RAG grounds LLM responses in your own documents by retrieving relevant context at query time and feeding it into the prompt.
Retrieval-Augmented Generation (RAG) Basics is a free Prompt Engineering & LLM Optimization for Developers lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Prompt Engineering & LLM Optimization for Developers learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
The Knowledge Gap
An LLM only knows what it was trained on. It cannot answer questions about your private docs or recent events. RAG (Retrieval-Augmented Generation) closes this gap by fetching relevant text and putting it in the prompt.
The Core Idea
Instead of fine-tuning the model on your data, you retrieve the most relevant snippets at query time and let the model answer using them as context. Cheaper, faster to update, and easy to cite.
Step 1: Chunking
Documents are split into small chunks (a few hundred tokens each). Chunks small enough to be precise, large enough to keep meaning.
chunks = split(document, size=500, overlap=50)Step 2: Embeddings
Each chunk is converted to a vector with an embedding model. Similar meanings produce nearby vectors, enabling semantic search.
vector = embed("Refunds are processed in 5 days.")
// -> [0.012, -0.43, 0.88, ...]Step 3: The Vector Store
Vectors are saved in a vector database like Pinecone, Weaviate, or pgvector. It supports fast nearest-neighbor search over millions of chunks.
Step 4: Retrieve at Query Time
When a user asks a question, you embed the question and find the top-k most similar chunks.
q = embed(userQuestion);
results = store.search(q, topK=4);Step 5: Augment the Prompt
The retrieved chunks are inserted into the prompt as context, and the model is told to answer using only that context.
Use only the context below to answer.
Context:
{retrieved_chunks}
Question: {user_question}Grounding and Citations
Because the answer is built from real chunks, you can show citations back to the source documents, and you can instruct the model to say I do not know when the context lacks the answer.
Why Not Just Fine-Tune?
- RAG updates instantly: change a doc, re-index, done.
- Fine-tuning is slow and bakes knowledge in.
- RAG gives traceable sources; fine-tuning does not.
Common RAG Problems
Poor chunking, weak embeddings, or retrieving too few chunks all hurt quality. If answers are wrong, inspect what was retrieved first; the issue is usually retrieval, not the model.
Improving Retrieval
- Add overlap between chunks.
- Use hybrid keyword + vector search.
- Re-rank results with a cross-encoder.
- Tune top-k for your context window.
Quick Check
Test your understanding of RAG.
Recap
RAG chunks documents, embeds them into a vector store, retrieves the most relevant chunks for each query, and augments the prompt. It grounds answers, enables citations, and stays current without retraining.
Frequently asked questions
Is the “Retrieval-Augmented Generation (RAG) Basics” lesson free?
Yes — the full text of “Retrieval-Augmented Generation (RAG) Basics” is free to read here on the web, and the Prompt Engineering & LLM Optimization for Developers course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Prompt Engineering & LLM Optimization for Developers course, upgrade to CoddyKit PRO.
What will I learn in “Retrieval-Augmented Generation (RAG) Basics”?
Learn how RAG grounds LLM responses in your own documents by retrieving relevant context at query time and feeding it into the prompt. You practise Prompt Engineering & LLM Optimization for Developers with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Prompt Engineering & LLM Optimization for Developers?
No prior experience is required. Prompt Engineering & LLM Optimization for Developers on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Retrieval-Augmented Generation (RAG) Basics” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Prompt Engineering & LLM Optimization for Developers lesson?
Yes. Every Prompt Engineering & LLM Optimization for Developers lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- LLM API Interaction (OpenAI, Anthropic)
- LangChain & LlamaIndex Basics
- Prompt Management & Versioning
- Retrieval-Augmented Generation (RAG) Basics