Vector Databases: Pinecone, Weaviate & pgvector · 课时

RAG 的文本块拆分策略

学习如何将文档拆分为适合嵌入的有效文本块，让 RAG 系统检索到精准且相关的上下文。

第 4 / 4 课13 个步骤

RAG 的文本块拆分策略 是 CoddyKit 上的免费 Vector Databases: Pinecone, Weaviate & pgvector 课时。这是第 4 节课，共 4 节。你可以在下方免费阅读本课时的完整内容 — 然后在浏览器中使用内置代码编辑器和全天候 AI 导师进行实践。这是 Vector Databases: Pinecone, Weaviate & pgvector 学习路径的一部分，你的进度在网页和 CoddyKit 应用中同步。 Vector Databases: Pinecone, Weaviate & pgvector 课程共包含 4 节课。

本课时的部分内容尚未翻译，以英文显示。

Why Chunking Matters

Before embedding, documents are split into chunks. Chunk quality directly determines retrieval quality — too large dilutes relevance, too small loses context.

Fixed-Size Chunking

The simplest method splits text every N characters or tokens. Fast but can cut sentences mid-thought.

def fixed_chunks(text, size=500):
    return [text[i:i+size] for i in range(0, len(text), size)]

Overlapping Windows

Add overlap between chunks so context spanning a boundary is not lost. A 10-20% overlap is common.

def overlap_chunks(text, size=500, overlap=100):
    step = size - overlap
    return [text[i:i+size] for i in range(0, len(text), step)]

Sentence-Aware Splitting

Split on sentence boundaries so each chunk stays grammatically whole. This usually improves embedding quality over raw character splits.

import re
sentences = re.split(r'(?<=[.!?])\s+', document_text)

Recursive Chunking

Recursive splitting tries large separators first (paragraphs), then smaller ones (sentences, words) until chunks fit the size limit. It respects natural structure.

Semantic Chunking

Semantic chunking groups sentences by meaning similarity, starting a new chunk when topic shifts. More expensive but very precise.

Chunk Size vs Model

Match chunk size to your embedding model's context window and your LLM's prompt budget.

Small chunks: precise but fragmented
Large chunks: rich but noisy

Keeping Metadata

Store source, page, and position metadata with each chunk so you can cite sources and reconstruct context later.

chunk = {
  'text': part,
  'source': 'guide.pdf',
  'page': 4,
  'position': idx
}

Parent-Child Chunks

Embed small child chunks for precise matching, but return their larger parent chunk to the LLM for full context. Best of both worlds.

Evaluating Chunking

Test different chunking configs with the same queries and measure retrieval hit rate. There is no universal best — measure for your data.

Practical Defaults

A solid starting point:

Recursive splitting
~500 tokens per chunk
~50-100 token overlap
Attach source metadata

Quick Check

Test your chunking knowledge.

在交互式课程中回答

Recap

You learned fixed, overlapping, sentence-aware, recursive, and semantic chunking, plus parent-child retrieval and metadata. Good chunking is the foundation of accurate RAG retrieval.

免费开始

用 AI 导师学习 Vector Databases: Pinecone, Weaviate & pgvector — 免费

在浏览器中编写并运行真实代码，获得全天候 AI 导师的即时帮助，并在网页或应用中继续学习。

课程: 12
课程: 48

免费开始学习查看完整课程

常见问题解答

「RAG 的文本块拆分策略」课时是免费的吗？

是的 — 「RAG 的文本块拆分策略」的完整文本可在网页上免费阅读。要进行交互式练习（内置代码编辑器和全天候 AI 导师）并解锁 Vector Databases: Pinecone, Weaviate & pgvector 课程的其余内容，请升级到 CoddyKit PRO。 Vector Databases: Pinecone, Weaviate & pgvector 课程共包含 4 节课。

「RAG 的文本块拆分策略」这节课中我会学到什么？

学习如何将文档拆分为适合嵌入的有效文本块，让 RAG 系统检索到精准且相关的上下文。你通过在浏览器中直接运行的动手代码来练习 Vector Databases: Pinecone, Weaviate & pgvector，全天候 AI 导师会在你学习这节课的过程中回答你的问题。

学习 Vector Databases: Pinecone, Weaviate & pgvector 需要有经验吗？

无需任何先前经验。CoddyKit 上的 Vector Databases: Pinecone, Weaviate & pgvector 课程适合初学者到高级学习者，你可以从这里开始或从头开始，按照自己的节奏学习。这是第 4 节课，共 4 节。

「RAG 的文本块拆分策略」课时需要多长时间？

大多数 CoddyKit 课程大约需要 5–10 分钟。每节课都很精短且互动，所以你能稳步进步，并在网页和应用中从离开的地方继续。

我能在这节 Vector Databases: Pinecone, Weaviate & pgvector 课中编写并运行代码吗？

能。每节 Vector Databases: Pinecone, Weaviate & pgvector 课都包含内置代码编辑器，你可以在浏览器中直接编写并运行真实代码，并获得即时 AI 反馈 — 无需本地设置。

此课程中的所有课时

← 返回 Vector Databases: Pinecone, Weaviate & pgvector