存储与检索嵌入向量
实现从文档片段生成嵌入向量，并将其存入向量数据库以便后续检索
存储与检索嵌入向量 是 CoddyKit 上的免费 LangChain / RAG / Vector DBs 课时。 这是第 3 节课，共 4 节。 你可以在下方免费阅读本课时的完整内容 — 然后在浏览器中使用内置代码编辑器和全天候 AI 导师进行实践。 这是 LangChain / RAG / Vector DBs 学习路径的一部分，你的进度在网页和 CoddyKit 应用中同步。 LangChain / RAG / Vector DBs 课程共包含 4 节课。
本课时的部分内容尚未翻译，以英文显示。
Storing & Retrieving Embeddings
Welcome to Lesson 3! In this lesson, we'll connect the dots between text chunks and vector databases.
You'll learn how to generate numerical representations (embeddings) from your document chunks and then store them efficiently in a vector database for quick and accurate retrieval.
Recap: Chunks & Embeddings
Before we dive in, let's quickly recap. From previous lessons, you know:
- Document Chunks: Large documents are split into smaller, manageable pieces to fit LLM context windows and improve retrieval granularity.
- Text Embeddings: These are numerical vectors that capture the semantic meaning of text. Similar texts have similar embeddings.
Our goal now is to turn those chunks into embeddings and make them searchable!
The Storage & Retrieval Flow
Here's the typical workflow for getting your data ready for RAG:
- Load & Split: Ingest raw documents and break them into chunks.
- Embed: Convert each text chunk into an embedding vector using an embedding model.
- Store: Save these embedding vectors (along with their original text chunks and metadata) in a vector database.
- Retrieve: When a user asks a question, embed the query, search the vector database for similar embeddings, and retrieve the most relevant chunks.
Initializing an Embedding Model
First, we need an embedding model. LangChain provides interfaces for many models, including those from OpenAI, Cohere, and local models like those from Hugging Face.
For this example, we'll use a local Hugging Face model to avoid needing an API key. This model turns text into a vector of numbers.
from langchain_community.embeddings import HuggingFaceEmbeddings
# Initialize a local embedding model
# This might download the model the first time
embeddings_model = HuggingFaceEmbeddings(model_name="sentence-transformers/all-MiniLM-L6-v2")
print("Embedding model initialized successfully!")Generating Embeddings from Text
Once our embedding model is ready, we can feed it text chunks. The model will then output a list of numbers (our embedding vector) for each chunk.
These vectors are what the vector database will use to find similar pieces of information.
from langchain_community.embeddings import HuggingFaceEmbeddings
embeddings_model = HuggingFaceEmbeddings(model_name="sentence-transformers/all-MiniLM-L6-v2")
texts_to_embed = [
"The quick brown fox jumps over the lazy dog.",
"A canine named Fido is taking a nap."
]
# Generate embeddings for the texts
vectors = embeddings_model.embed_documents(texts_to_embed)
print(f"Number of vectors generated: {len(vectors)}")
print(f"Dimension of each vector: {len(vectors[0])}")
# print(f"First vector (partial): {vectors[0][:5]}...") # Too long for mobile
Introducing Vector Stores
A vector store is a specialized database designed to efficiently store and query high-dimensional vectors. It's built to quickly find vectors that are 'close' to a given query vector.
Think of it as a super-fast index for semantic meaning. When you search, it doesn't look for keywords; it looks for meaning.
LangChain supports many vector stores, from simple in-memory ones like FAISS to robust cloud services like Pinecone or Chroma.
Storing Embeddings with FAISS
Let's use FAISS, an in-memory vector store, to demonstrate storage. We'll take our text chunks and their embeddings and add them to FAISS. FAISS handles the indexing for fast search.
In a real application, you'd load chunks from documents first, then embed them, and finally store them. Here, we'll create simple documents directly.
from langchain_community.vectorstores import FAISS
from langchain_community.embeddings import HuggingFaceEmbeddings
from langchain_core.documents import Document
embeddings_model = HuggingFaceEmbeddings(model_name="sentence-transformers/all-MiniLM-L6-v2")
# Create some example documents (text chunks with optional metadata)
documents = [
Document(page_content="The capital of France is Paris.", metadata={"source": "geo"}),
Document(page_content="Eiffel Tower is a landmark in Paris.", metadata={"source": "tourism"}),
Document(page_content="Python is a popular programming language.", metadata={"source": "tech"}),
Document(page_content="Coding with Python is fun and versatile.", metadata={"source": "tech"})
]
# Create a FAISS vector store from the documents and embeddings model
vectorstore = FAISS.from_documents(documents, embeddings_model)
print("Documents successfully stored in FAISS vector store!")Performing a Similarity Search
Now that our documents are embedded and stored, we can query the vector store to find the most semantically similar documents to our question.
The vector store will embed your query, compare its vector to all stored vectors, and return the top 'k' (e.g., top 4) most similar documents.
from langchain_community.vectorstores import FAISS
from langchain_community.embeddings import HuggingFaceEmbeddings
from langchain_core.documents import Document
embeddings_model = HuggingFaceEmbeddings(model_name="sentence-transformers/all-MiniLM-L6-v2")
documents = [
Document(page_content="The capital of France is Paris.", metadata={"source": "geo"}),
Document(page_content="Eiffel Tower is a landmark in Paris.", metadata={"source": "tourism"}),
Document(page_content="Python is a popular programming language.", metadata={"source": "tech"}),
Document(page_content="Coding with Python is fun and versatile.", metadata={"source": "tech"})
]
vectorstore = FAISS.from_documents(documents, embeddings_model)
query = "What is the main city of France?"
# Perform a similarity search
retrieved_docs = vectorstore.similarity_search(query, k=2)
print(f"Query: '{query}'\n")
print("Top 2 retrieved documents:")
for i, doc in enumerate(retrieved_docs):
print(f"{i+1}. Content: '{doc.page_content}' (Source: {doc.metadata['source']})")Metadata is Your Friend
Notice in the previous example how we included metadata with our documents? This is incredibly powerful!
- Filtering: You can filter searches based on metadata (e.g., only search documents from a specific author or date).
- Context: Metadata helps the LLM understand the source and relevance of the retrieved chunk, improving answer quality.
- Debugging: It's easier to trace where information came from.
Always consider what metadata is useful to store alongside your text chunks.
Quick Check
You've learned about the steps to prepare your data for a RAG system. What is the correct sequence for storing and retrieving information?
Recap: Storing & Retrieving
Great job! In this lesson, you've mastered the critical steps of preparing your data for a RAG system:
- Initializing an embedding model to convert text into numerical vectors.
- Understanding the role of a vector store for efficient storage and similarity search.
- Implementing the process of generating embeddings and storing them (e.g., using FAISS).
- Performing similarity searches to retrieve relevant document chunks based on a query.
- Recognizing the importance of metadata for richer context and filtering.
These skills are fundamental to building effective RAG applications!
用 AI 导师学习 LangChain / RAG / Vector DBs — 免费
在浏览器中编写并运行真实代码，获得全天候 AI 导师的即时帮助，并在网页或应用中继续学习。
- 课程
- 12
- 课程
- 48
常见问题解答
「存储与检索嵌入向量」课时是免费的吗？
是的 — 「存储与检索嵌入向量」的完整文本可在网页上免费阅读。要进行交互式练习（内置代码编辑器和全天候 AI 导师）并解锁 LangChain / RAG / Vector DBs 课程的其余内容，请升级到 CoddyKit PRO。 LangChain / RAG / Vector DBs 课程共包含 4 节课。
「存储与检索嵌入向量」这节课中我会学到什么？
实现从文档片段生成嵌入向量，并将其存入向量数据库以便后续检索 你通过在浏览器中直接运行的动手代码来练习 LangChain / RAG / Vector DBs，全天候 AI 导师会在你学习这节课的过程中回答你的问题。
学习 LangChain / RAG / Vector DBs 需要有经验吗？
无需任何先前经验。CoddyKit 上的 LangChain / RAG / Vector DBs 课程适合初学者到高级学习者，你可以从这里开始或从头开始，按照自己的节奏学习。 这是第 3 节课，共 4 节。
「存储与检索嵌入向量」课时需要多长时间？
大多数 CoddyKit 课程大约需要 5–10 分钟。每节课都很精短且互动，所以你能稳步进步，并在网页和应用中从离开的地方继续。
我能在这节 LangChain / RAG / Vector DBs 课中编写并运行代码吗？
能。每节 LangChain / RAG / Vector DBs 课都包含内置代码编辑器，你可以在浏览器中直接编写并运行真实代码，并获得即时 AI 反馈 — 无需本地设置。