Memory and State in Agentic RAG
Give your RAG agent short-term and long-term memory so it can hold conversations and recall facts across turns.
Memory and State in Agentic RAG is a free LangChain / RAG / Vector DBs lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the LangChain / RAG / Vector DBs learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Why Agents Need Memory
A stateless agent forgets everything after each call. Memory lets an agent track the conversation, remember user preferences, and avoid repeating retrievals.
Short-Term vs. Long-Term
Short-term memory holds the current conversation in the context window. Long-term memory persists facts across sessions, often in a vector store or database.
Conversation Buffer
The simplest memory keeps the full message history and replays it each turn so the model has full context.
history = []
def chat(user_msg):
history.append({"role": "user", "content": user_msg})
reply = llm.invoke(history)
history.append({"role": "assistant", "content": reply})
return replyThe Context Window Limit
Replaying the entire history eventually overflows the model context. You must summarize or trim older turns to stay within the token budget.
Summary Memory
Periodically compress old turns into a running summary, keeping recent turns verbatim. This preserves gist while freeing tokens.
if len(history) > 20:
summary = llm.invoke("Summarize: " + str(history[:-6]))
history[:] = [{"role": "system", "content": summary}] + history[-6:]Long-Term Memory via Vectors
Store durable facts as embeddings. On each turn, retrieve relevant memories by similarity and inject them, just like RAG over a knowledge base.
memory_store.add_texts(["User prefers metric units"])
relevant = memory_store.similarity_search(user_msg, k=3)Choosing What to Remember
Do not store everything. Extract durable, reusable facts (preferences, decisions, entities) and skip ephemeral chatter to keep long-term memory clean.
State Beyond Chat
Agents also track non-conversational state: which tools ran, intermediate results, and a scratchpad of reasoning steps used to plan the next action.
Thread and Session Keys
In multi-user systems, scope memory by a thread_id or user_id so conversations never leak between people.
def get_history(thread_id):
return store.get(thread_id, [])
def save(thread_id, messages):
store[thread_id] = messagesPersistence
For memory to survive restarts, back it with a database or checkpointer rather than an in-process dictionary. LangGraph offers checkpointers for exactly this.
Putting It Together
Combine a trimmed conversation buffer for recency, summary memory for the middle, and vector long-term memory for durable facts, all keyed by session.
Quick Check
Test your understanding of agent memory.
Recap
You added memory to agents:
- Short-term buffer for the current chat
- Summary memory to fit the context window
- Long-term vector memory for durable facts
- Scope by session and persist for durability
Frequently asked questions
Is the “Memory and State in Agentic RAG” lesson free?
Yes — the full text of “Memory and State in Agentic RAG” is free to read here on the web, and the LangChain / RAG / Vector DBs course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the LangChain / RAG / Vector DBs course, upgrade to CoddyKit PRO.
What will I learn in “Memory and State in Agentic RAG”?
Give your RAG agent short-term and long-term memory so it can hold conversations and recall facts across turns. You practise LangChain / RAG / Vector DBs with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start LangChain / RAG / Vector DBs?
No prior experience is required. LangChain / RAG / Vector DBs on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Memory and State in Agentic RAG” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this LangChain / RAG / Vector DBs lesson?
Yes. Every LangChain / RAG / Vector DBs lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- LangChain Agents and Tool Concepts
- Building Multi-Agent RAG Workflows
- Integrating External APIs as Tools
- Memory and State in Agentic RAG