Prompt Engineering & Context Windows
Understand the context window that bounds every LLM call and learn to craft prompts that fit retrieved context, instructions, and questions together for reliable production answers.
Prompt Engineering & Context Windows is a free LLM Apps in Production (RAG + Vector DB + Caching) lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the LLM Apps in Production (RAG + Vector DB + Caching) learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
The Context Window
Every LLM has a fixed context window — the max tokens it reads and writes per call. System prompt, retrieved docs, history, and your question all have to fit.
What Lives in the Window
A production RAG prompt packs in system instructions, retrieved context, prior history, and the current question. Exceed the window and something gets cut.
Anatomy of a Prompt
A clear prompt structure helps the model tell instructions apart from data. Here's a clean layout separating context from the question.
prompt = (
'You are a support assistant. '
'Answer ONLY from the context.\n\n'
'Context:\n{context}\n\n'
'Question: {question}'
)Grounding Instructions
To cut hallucination, add grounding instructions: tell the model to answer only from the provided context and to admit when it doesn't know.
rule = 'If the answer is not in the context, say you do not know.'Using a Prompt Template
A prompt template makes prompts reusable and safe to fill with variables. LangChain's ChatPromptTemplate does exactly this.
from langchain_core.prompts import ChatPromptTemplate
template = ChatPromptTemplate.from_messages([
('system', 'Answer only from context: {context}'),
('human', '{question}')
])Counting Tokens
Before sending, count tokens so you don't overflow the window. Rough rule for English: about 4 characters per token.
import tiktoken
enc = tiktoken.get_encoding('cl100k_base')
print(len(enc.encode(filled_prompt)))When Context Is Too Big
When context is too big, shrink it: fewer chunks, smaller chunk sizes, or summarize. Never silently truncate the middle — you might drop the answer.
The Lost-in-the-Middle Effect
Watch the lost-in-the-middle effect: models attend best to the start and end of context, worst to the middle. Put your most relevant chunks first or last.
Reserving Output Space
Input and output share the window. Fill it all with input and there's no room to generate — so reserve output space for the expected answer length.
max_output = 800
budget_for_input = WINDOW - max_outputFew-Shot Examples
A few few-shot examples can steer format and tone, but they eat tokens. Weigh their value against the space they consume.
Iterating on Prompts
Prompt engineering is empirical: change one thing at a time, test on real questions, and measure. Small wording tweaks can shift answer quality a lot.
Quick Check
Test your understanding of context windows.
Recap
Recap: the context window holds system, context, history, question, and output. Structure prompts, ground them, count tokens, beat lost-in-the-middle, and reserve output room.
Frequently asked questions
Is the “Prompt Engineering & Context Windows” lesson free?
Yes — the full text of “Prompt Engineering & Context Windows” is free to read here on the web, and the LLM Apps in Production (RAG + Vector DB + Caching) course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the LLM Apps in Production (RAG + Vector DB + Caching) course, upgrade to CoddyKit PRO.
What will I learn in “Prompt Engineering & Context Windows”?
Understand the context window that bounds every LLM call and learn to craft prompts that fit retrieved context, instructions, and questions together for reliable production answers. You practise LLM Apps in Production (RAG + Vector DB + Caching) with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start LLM Apps in Production (RAG + Vector DB + Caching)?
No prior experience is required. LLM Apps in Production (RAG + Vector DB + Caching) on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Prompt Engineering & Context Windows” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this LLM Apps in Production (RAG + Vector DB + Caching) lesson?
Yes. Every LLM Apps in Production (RAG + Vector DB + Caching) lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Understanding LLM Apps in Production
- Fundamentals of Retrieval Augmented Generation
- Basic RAG System Architecture Overview
- Prompt Engineering & Context Windows