AI Agents · Lesson

A Q&A Bot Over Your Documents

Ship a working RAG bot: ingest PDFs, embed, search, generate. The 'hello world' of production agents.

A Q&A Bot Over Your Documents is a free AI Agents lesson on CoddyKit — lesson 1 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Agents learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

Project Goal

Build a chatbot that answers questions from your own documents (PDFs, Markdown, etc.). This is the "hello world" of production agents and immediately useful for any team.

Architecture

Ingest: load docs -> chunk -> embed -> store in vector DB
Query: embed question -> retrieve top-K chunks -> stuff in prompt -> LLM
Cite: format chunks with IDs; ask LLM to cite

Step 1: Load and Chunk

from langchain.text_splitter import RecursiveCharacterTextSplitter
from pypdf import PdfReader

text = ''
for page in PdfReader('handbook.pdf').pages:
    text += page.extract_text() + '\n'

splitter = RecursiveCharacterTextSplitter(chunk_size=800, chunk_overlap=100)
chunks = splitter.split_text(text)
print(f'{len(chunks)} chunks')

Step 2: Embed and Store

import chromadb
from openai import OpenAI

oai = OpenAI()
chroma = chromadb.PersistentClient(path='./db')
collection = chroma.get_or_create_collection('handbook')

resp = oai.embeddings.create(model='text-embedding-3-small', input=chunks)
vectors = [d.embedding for d in resp.data]
collection.add(ids=[f'c{i}' for i in range(len(chunks))], embeddings=vectors, documents=chunks)

Step 3: Query Function

def query(question, k=4):
    qvec = oai.embeddings.create(model='text-embedding-3-small', input=question).data[0].embedding
    res = collection.query(query_embeddings=[qvec], n_results=k)
    return res['documents'][0]

Step 4: Build the Prompt

def make_prompt(question, chunks):
    context = '\n\n'.join(f'[{i+1}] {c}' for i, c in enumerate(chunks))
    return f'''
Using only the context below, answer the question.
If the answer is not present, say so.
Cite the source like [1], [2], etc.

Context:
{context}

Question: {question}
Answer:
'''

print(make_prompt("What is the return policy?", ["Returns accepted within 30 days.", "Item must be unused."]))

Step 5: Generate

def answer(question):
    chunks = query(question)
    prompt = make_prompt(question, chunks)
    response = oai.chat.completions.create(
        model='gpt-4o-mini',
        messages=[{'role': 'user', 'content': prompt}],
        temperature=0,
    )
    return response.choices[0].message.content

print(answer('What is our refund policy?'))

Wire It Up to a UI

Wrap in FastAPI + a simple HTML page (or Streamlit) and you have a working app.

# pip install fastapi uvicorn
from fastapi import FastAPI
app = FastAPI()

@app.post('/ask')
def ask(payload: dict):
    return {'answer': answer(payload['question'])}

Add Chat History

For multi-turn Q&A, keep messages and pass them into the prompt:

messages = [{'role': 'system', 'content': 'You are a doc Q&A assistant.'}]

def chat(user_input):
    messages.append({'role': 'user', 'content': user_input})
    chunks = query(user_input)
    context_msg = {'role': 'system', 'content': f'Context:\n{chr(10).join(chunks)}'}
    response = oai.chat.completions.create(
        model='gpt-4o-mini',
        messages=[messages[0], context_msg] + messages[1:]
    )
    answer = response.choices[0].message.content
    messages.append({'role': 'assistant', 'content': answer})
    return answer

Common Pitfalls

Chunks too big or too small — try 500-800 tokens
No source citation — users do not trust uncited answers
K too large — wastes tokens, dilutes attention
Embedding the WHOLE question including chat history — embed just the latest user question

Add Source Links

Store source URLs in metadata and surface them in the answer:

metadata = [{'source': 'handbook.pdf', 'page': i} for i in range(len(chunks))]
# In the prompt, include 'source: <url>' so the LLM can produce clickable references.

Evaluate It

Gather 20 real questions from users. Manually mark whether the bot answers correctly. Use this as your starting eval set.

Ship and Iterate

This bot, in production, handles 80% of common questions. Iterate based on the 20% it gets wrong — usually you need better chunking or re-ranking.

Source Citation

Why include source IDs in the prompt and ask the LLM to cite?

Recap

RAG bot in ~50 lines. Ship it, evaluate it, iterate on chunking and retrieval. This pattern repeats across countless agent apps.

Frequently asked questions

Is the “A Q&A Bot Over Your Documents” lesson free?

Yes — the full text of “A Q&A Bot Over Your Documents” is free to read here on the web, and the AI Agents course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Agents course, upgrade to CoddyKit PRO.

What will I learn in “A Q&A Bot Over Your Documents”?

Ship a working RAG bot: ingest PDFs, embed, search, generate. The 'hello world' of production agents. You practise AI Agents with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start AI Agents?

No prior experience is required. AI Agents on CoddyKit is structured for beginners through advanced learners; this is — lesson 1 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “A Q&A Bot Over Your Documents” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this AI Agents lesson?

Yes. Every AI Agents lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

← Back to AI Agents