使用 IVFFlat 索引提升速度
在 pgvector 中实现 IVFFlat 索引,加速近似最近邻搜索,让查询更快。
使用 IVFFlat 索引提升速度 是 CoddyKit 上的免费 Vector Databases: Pinecone, Weaviate & pgvector 课时。 这是第 1 节课,共 4 节。 你可以在下方免费阅读本课时的完整内容 — 然后在浏览器中使用内置代码编辑器和全天候 AI 导师进行实践。 这是 Vector Databases: Pinecone, Weaviate & pgvector 学习路径的一部分,你的进度在网页和 CoddyKit 应用中同步。 Vector Databases: Pinecone, Weaviate & pgvector 课程共包含 4 节课。
本课时的部分内容尚未翻译,以英文显示。
Faster Searches with IVFFlat
Welcome to optimizing pgvector performance! For small datasets, a simple vector similarity query might be fast enough. But as your data grows, brute-force search becomes too slow.
This lesson introduces IVFFlat indexing, a powerful technique to accelerate Approximate Nearest Neighbor (ANN) searches in pgvector.
Why Vector Indexes are Key
Imagine searching for the closest person to you in a massive crowd. Without any organization, you'd have to check every single person.
Similarly, without an index, pgvector has to compare your query vector to every single vector in your table. This is called a brute-force search, and it's inefficient for large datasets.
- Indexes organize data.
- They make lookups much faster.
- Crucial for scalable vector search.
IVFFlat: The Core Idea
IVFFlat (Inverted File Index with Flat quantizer) is an ANN indexing algorithm. Its core idea is to group similar vectors into 'lists' or 'clusters'.
Think of it like sorting books by genre before looking for a specific title. You first pick the right genre, then search within that smaller section.
How IVFFlat Works: Two Steps
When you query an IVFFlat index, it performs a two-step process:
- Find nearest lists: It quickly identifies a few (or more) clusters that are closest to your query vector.
- Search within lists: It then only searches for nearest neighbors within those selected clusters, ignoring the rest of the dataset.
This significantly reduces the number of comparisons needed, speeding up your queries.
The `lists` Parameter Explained
When creating an IVFFlat index, the most important parameter is lists. This defines how many clusters (or partitions) your data will be divided into.
- More
lists: Each cluster has fewer vectors. This can lead to faster searches (less data to scan per cluster). - Fewer
lists: Each cluster has more vectors. Searches might be slower, but you're less likely to miss true nearest neighbors.
It's a trade-off between search speed and recall (how many of the true nearest neighbors you actually find).
Prepare Your Vector Table
Before creating an index, you need a table with a vector column. If you haven't already, ensure the vector extension is enabled.
Here's a quick setup for a table with 3-dimensional vectors:
CREATE EXTENSION IF NOT EXISTS vector;
CREATE TABLE items (
id serial PRIMARY KEY,
embedding vector(3)
);
INSERT INTO items (embedding) VALUES
('[1,2,3]'),
('[1.1,2.1,3.1]'),
('[4,5,6]'),
('[4.1,5.1,6.1]'),
('[7,8,9]'),
('[7.1,8.1,9.1]');Creating an IVFFlat Index
Now, let's create an IVFFlat index on our embedding column. We'll specify the vector_l2_ops operator class for L2 distance (Euclidean distance), and set the lists parameter.
A good starting point for lists is rows / 1000 for up to 1M rows, or sqrt(rows) for larger datasets.
CREATE INDEX ON items USING ivfflat (embedding vector_l2_ops) WITH (lists = 4);Querying with the Index
Once the IVFFlat index is created, your standard similarity queries will automatically benefit from it! pgvector will use the index to find approximate nearest neighbors much faster.
Let's find the 3 closest items to [1,2,3]:
SELECT id, embedding <-> '[1,2,3]' AS distance
FROM items
ORDER BY distance
LIMIT 3;The `probes` Query Parameter
At query time, you can further tune the search with the ivfflat.probes parameter. This setting controls how many of the nearest 'lists' (clusters) are actually searched.
- More
probes: Better recall, but slower search. - Fewer
probes: Faster search, but potentially lower recall.
The default probes value is 1. You can set it for your session:
SET ivfflat.probes = 2;
SELECT id, embedding <-> '[1,2,3]' AS distance
FROM items
ORDER BY distance
LIMIT 3;IVFFlat Indexing Check
Let's test your understanding of IVFFlat indexing.
IVFFlat: Key Takeaways
Great job! You've learned the fundamentals of IVFFlat indexing in pgvector.
- IVFFlat speeds up ANN searches by partitioning data.
- The
listsparameter (index creation) controls the number of clusters. - The
ivfflat.probesparameter (query time) controls how many clusters are searched. - Both involve a trade-off between search speed and recall.
Next, we'll explore HNSW indexing, another powerful option for even higher recall.
常见问题解答
「使用 IVFFlat 索引提升速度」课时是免费的吗?
是的 — 「使用 IVFFlat 索引提升速度」的完整文本可在网页上免费阅读。要进行交互式练习(内置代码编辑器和全天候 AI 导师)并解锁 Vector Databases: Pinecone, Weaviate & pgvector 课程的其余内容,请升级到 CoddyKit PRO。 Vector Databases: Pinecone, Weaviate & pgvector 课程共包含 4 节课。
「使用 IVFFlat 索引提升速度」这节课中我会学到什么?
在 pgvector 中实现 IVFFlat 索引,加速近似最近邻搜索,让查询更快。 你通过在浏览器中直接运行的动手代码来练习 Vector Databases: Pinecone, Weaviate & pgvector,全天候 AI 导师会在你学习这节课的过程中回答你的问题。
学习 Vector Databases: Pinecone, Weaviate & pgvector 需要有经验吗?
无需任何先前经验。CoddyKit 上的 Vector Databases: Pinecone, Weaviate & pgvector 课程适合初学者到高级学习者,你可以从这里开始或从头开始,按照自己的节奏学习。 这是第 1 节课,共 4 节。
「使用 IVFFlat 索引提升速度」课时需要多长时间?
大多数 CoddyKit 课程大约需要 5–10 分钟。每节课都很精短且互动,所以你能稳步进步,并在网页和应用中从离开的地方继续。
我能在这节 Vector Databases: Pinecone, Weaviate & pgvector 课中编写并运行代码吗?
能。每节 Vector Databases: Pinecone, Weaviate & pgvector 课都包含内置代码编辑器,你可以在浏览器中直接编写并运行真实代码,并获得即时 AI 反馈 — 无需本地设置。
此课程中的所有课时
- 使用 IVFFlat 索引提升速度
- 使用 HNSW 索引提升召回率
- 查询性能调优
- 过滤搜索优化