AI Prompt Engineering · Lesson

Vector Databases for Prompting

Explore how vector databases store and retrieve relevant information to feed into RAG prompts effectively.

Vector Databases for Prompting is a free AI Prompt Engineering lesson on CoddyKit — lesson 3 of 3. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Prompt Engineering learning path, one of 3 lessons in the course, and your progress syncs across the web and the CoddyKit app.

What are Vector Databases?

Welcome to the world of Vector Databases (VDBs)! These specialized databases are game-changers for working with Large Language Models (LLMs).

VDBs store information as numerical vectors, which are just lists of numbers. This allows them to quickly find data that is "semantically similar" – meaning it has a similar underlying meaning, not just matching keywords.

Language as Numbers: Embeddings

Before data enters a VDB, it's transformed into an embedding. An embedding is a numerical representation of text, where words or phrases with similar meanings are close together in a multi-dimensional space.

Text to Vector: An embedding model converts your text into a vector.
Meaning Preserved: These vectors capture the semantic meaning of the text.
LLM's Language: This is how LLMs "understand" and process language internally.

Storing Data in a Vector Database

Think of a vector database as a highly organized library where every book (or piece of text) is tagged with a unique numerical fingerprint – its embedding vector.

When you add data to a VDB, the process looks like this:

Your text is broken into smaller chunks (if needed).
Each chunk is converted into an embedding vector.
The VDB stores this vector along with its original text and any associated metadata.

The Magic of Similarity Search

The real power of VDBs lies in their ability to perform similarity search. Instead of keyword matching, they find vectors that are numerically "close" to your query vector.

This means if you ask about "canine companions," a VDB can retrieve documents mentioning "dogs" or "pets," even if those exact words aren't present. It understands the underlying meaning!

VDBs in Retrieval Augmented Generation (RAG)

Vector databases are the core component of the "Retrieval" step in a RAG system. They act as the external knowledge base that LLMs can tap into.

Here's the simplified flow:

User asks a question.
The question is embedded into a vector.
The VDB finds the most relevant document chunks (vectors).
These chunks are sent to the LLM as context to answer the question.

Populating a VDB: Illustrative Code

Let's look at a conceptual Python example of how you might add documents to a vector database. This process involves embedding your text and then storing the resulting vectors.

Remember, this code is illustrative and requires specific vector database client libraries and an embedding model in a real-world setup.

import numpy as np

# Imagine an embedding model function
def get_embedding(text):
    # In a real scenario, this uses an LLM API
    # or a local embedding model.
    # For illustration, let's return a dummy vector.
    return np.random.rand(128).tolist() # 128-dim vector

# Imagine a simplified VectorDB client
class SimpleVectorDB:
    def __init__(self):
        self.vectors = {} # Stores {id: {'vector': [...], 'text': '...'}}

    def add_document(self, doc_id, text_content):
        vector = get_embedding(text_content)
        self.vectors[doc_id] = {'vector': vector, 'text': text_content}
        print(f"Added '{text_content[:20]}...' (ID: {doc_id})")

# --- Main simulation --- 
db = SimpleVectorDB()
db.add_document("doc1", "The capital of France is Paris.")
db.add_document("doc2", "Eiffel Tower is a landmark in Paris.")
db.add_document("doc3", "Berlin is the capital of Germany.")

Querying a VDB: Illustrative Code

Once your vector database is populated, you can query it to find relevant information. The query itself is also embedded, and then the VDB searches for the closest vectors.

This conceptual Python code demonstrates searching for semantically similar content.

import numpy as np
from scipy.spatial.distance import cosine # For similarity

# (Re-using get_embedding and SimpleVectorDB conceptually)
def get_embedding(text):
    return np.random.rand(128).tolist()

class SimpleVectorDB:
    def __init__(self):
        self.vectors = {}

    def add_document(self, doc_id, text_content):
        vector = get_embedding(text_content)
        self.vectors[doc_id] = {'vector': vector, 'text': text_content}

    def search(self, query_text, top_k=1):
        query_vector = get_embedding(query_text)
        
        scores = []
        for doc_id, data in self.vectors.items():
            doc_vector = data['vector']
            # Cosine similarity: 1 - cosine distance
            similarity = 1 - cosine(query_vector, doc_vector)
            scores.append({'id': doc_id, 'text': data['text'], 'score': similarity})
        
        # Sort by similarity in descending order
        scores.sort(key=lambda x: x['score'], reverse=True)
        return scores[:top_k]

# --- Main simulation --- 
db = SimpleVectorDB()
db.add_document("doc1", "The capital of France is Paris.")
db.add_document("doc2", "Eiffel Tower is a landmark in Paris.")
db.add_document("doc3", "Berlin is the capital of Germany.")

user_query = "What is the capital city of France?"
results = db.search(user_query, top_k=2)

print(f"Query: '{user_query}'")
print("Top results:")
for res in results:
    print(f"- Text: '{res['text']}' (Score: {res['score']:.2f})")

Key Benefits of Using VDBs

Integrating vector databases into your RAG workflow offers significant advantages:

Semantic Search: Finds results based on meaning, not just keywords.
Reduced Hallucinations: Grounds LLMs with factual, external data, making responses more reliable.
Scalability: Efficiently handles vast amounts of unstructured data.
Real-time Updates: Can be updated easily with new information, keeping your LLM context fresh.

Important Considerations for VDBs

To get the best performance from your vector database setup, consider these points:

Embedding Model Choice: The quality of your embeddings directly impacts search accuracy.
Chunking Strategy: How you break down documents (e.g., by paragraph, sentence) affects retrieval granularity.
Data Freshness: Regularly update your VDB with new information to ensure the LLM has the latest context.
Metadata: Store useful metadata alongside vectors for filtering and better retrieval.

Quick Check: Vector DBs in RAG

Vector databases play a critical role in enhancing LLM performance and reliability. Let's test your understanding of their primary function within a Retrieval Augmented Generation (RAG) system.

Recap: VDBs and Your LLMs

In this lesson, we explored vector databases and their vital role in modern AI applications, especially with RAG.

VDBs store numerical embeddings that capture the meaning of text.
They enable powerful semantic search, finding information based on context.
VDBs significantly enhance LLMs by providing external, factual data, leading to more accurate and reliable outputs.

Mastering VDBs is key to building robust and intelligent AI systems!

Frequently asked questions

Is the “Vector Databases for Prompting” lesson free?

Yes — the full text of “Vector Databases for Prompting” is free to read here on the web, and the AI Prompt Engineering course includes 3 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Prompt Engineering course, upgrade to CoddyKit PRO.

What will I learn in “Vector Databases for Prompting”?

Explore how vector databases store and retrieve relevant information to feed into RAG prompts effectively. You practise AI Prompt Engineering with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start AI Prompt Engineering?

No prior experience is required. AI Prompt Engineering on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 3, so you can start here or from the beginning and move at your own pace.

How long does the “Vector Databases for Prompting” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this AI Prompt Engineering lesson?

Yes. Every AI Prompt Engineering lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

← Back to AI Prompt Engineering