0Pricing
AI Agents · Lesson

Loaders, Splitters and Vector Stores

Use DocumentLoaders for PDFs/HTML, TextSplitters for chunking, and VectorStores for retrieval.

Loaders, Splitters and Vector Stores is a free AI Agents lesson on CoddyKit — lesson 2 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Agents learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

Three Pillars of RAG in LangChain

  1. Document Loaders — read raw data into Documents
  2. Text Splitters — chunk Documents
  3. Vector Stores — embed and index chunks

Document Loaders

LangChain has 100+ loaders. Examples:

from langchain_community.document_loaders import PyPDFLoader, WebBaseLoader, TextLoader

pdf_docs = PyPDFLoader('handbook.pdf').load()
web_docs = WebBaseLoader('https://example.com').load()
md_docs = TextLoader('README.md').load()

The Document Object

Every loader returns a list of Documents:

doc = pdf_docs[0]
print(doc.page_content)   # the text
print(doc.metadata)       # {'source': 'handbook.pdf', 'page': 1}

Common Loaders

  • PyPDFLoader, UnstructuredFileLoader — PDFs
  • WebBaseLoader, SitemapLoader — web pages
  • NotionLoader, GoogleDriveLoader — SaaS
  • GitLoader — code repos
  • SQLDatabaseLoader — DB rows

Text Splitters

Convert Documents to smaller chunks:

from langchain.text_splitter import RecursiveCharacterTextSplitter

splitter = RecursiveCharacterTextSplitter(
    chunk_size=800,
    chunk_overlap=100,
    separators=['\n\n', '\n', '. ', ' ']
)
chunks = splitter.split_documents(pdf_docs)

Specialised Splitters

  • MarkdownHeaderTextSplitter — splits by header
  • RecursiveCharacterTextSplitter — paragraph/sentence aware
  • HTMLHeaderTextSplitter — HTML-aware
  • Language-specific (Python, Markdown, LaTeX)

Token-Aware Splitting

Use the model's tokenizer for token-precise splits:

splitter = RecursiveCharacterTextSplitter.from_tiktoken_encoder(
    encoding_name='cl100k_base',
    chunk_size=400,
    chunk_overlap=50
)

Vector Stores

Embed and store the chunks:

from langchain_openai import OpenAIEmbeddings
from langchain_community.vectorstores import Chroma

embeddings = OpenAIEmbeddings(model='text-embedding-3-small')
store = Chroma.from_documents(
    documents=chunks,
    embedding=embeddings,
    persist_directory='./db'
)

Retrieval

retriever = store.as_retriever(search_kwargs={'k': 4})
results = retriever.invoke('What is our return policy?')
for d in results:
    print(d.page_content[:200])

Persistence

Chroma persists to disk if you give it persist_directory. To reload:

store = Chroma(persist_directory='./db', embedding_function=embeddings)

Other Vector Stores

Same interface, different backend:

from langchain_community.vectorstores import FAISS, Pinecone, Qdrant, Weaviate

store = FAISS.from_documents(chunks, embeddings)
store.save_local('./faiss_db')

MultiVectorRetriever

Index small chunks but retrieve large parents:

from langchain.retrievers import MultiVectorRetriever
# Use the parent-child pattern automatically.

Self-Query Retrievers

Let the LLM generate metadata filters from natural language:

from langchain.retrievers.self_query.base import SelfQueryRetriever
# 'What did Alice say in 2023?' -> filter={author: 'Alice', year: 2023}

Retriever Output

What does a LangChain retriever return?

Recap

Loaders + Splitters + Vector Stores = a full RAG stack. Next: composing them with LCEL into a complete chain.

Frequently asked questions

Is the “Loaders, Splitters and Vector Stores” lesson free?

Yes — the full text of “Loaders, Splitters and Vector Stores” is free to read here on the web, and the AI Agents course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Agents course, upgrade to CoddyKit PRO.

What will I learn in “Loaders, Splitters and Vector Stores”?

Use DocumentLoaders for PDFs/HTML, TextSplitters for chunking, and VectorStores for retrieval. You practise AI Agents with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start AI Agents?

No prior experience is required. AI Agents on CoddyKit is structured for beginners through advanced learners; this is — lesson 2 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “Loaders, Splitters and Vector Stores” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this AI Agents lesson?

Yes. Every AI Agents lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

  1. LangChain Architecture: Models, Prompts, Chains
  2. Loaders, Splitters and Vector Stores
  3. LCEL (LangChain Expression Language)
  4. Building a RAG Chain End-to-End
← Back to AI Agents