Por qué necesita una base de datos vectorial

NumPy semantic search works well for small corpora, but it has a fundamental scalability problem: every search scans every vector. At 1 million documents, each query requires 1.5 billion floating-point multiplications, taking hundreds of milliseconds. Worse, all vectors must fit in RAM.

Production AI systems need search over millions of documents in under 50ms. This is what vector databases are designed to deliver.

Approximate Nearest Neighbor (ANN) algorithms trade a small amount of accuracy for dramatic speed improvements. Instead of checking every vector, ANN algorithms use smart indexing structures to skip large portions of the search space.

In practice, ANN returns the true nearest neighbor over 95% of the time while being 100-1000x faster than exact search. For RAG, this tradeoff is almost always worth it.

HNSW (Hierarchical Navigable Small World) is the dominant ANN algorithm used by Pinecone, Weaviate, Qdrant, and pgvector. It builds a multi-layer graph where each node connects to its nearest neighbors. Search starts at the top sparse layer, navigates to the approximate region, then descends to the dense bottom layer for precision.

HNSW offers excellent query speed (logarithmic in dataset size) and high recall, but requires building the index upfront.

A vector database is more than an ANN index. It also provides:

• Metadata filtering — retrieve only vectors where category='finance' or date > '2024-01-01'
• Persistent storage — data survives restarts and scales beyond RAM
• CRUD operations — insert, update, and delete individual vectors
• Namespace isolation — separate collections for different customers or environments
• Horizontal scaling — distribute millions of vectors across shards

Metadata filtering lets you restrict retrieval to a relevant subset before running ANN search. For example, in a multi-tenant RAG system, you would filter by tenant_id so users only see their own documents. Without metadata filtering, you would need a separate index per tenant.

This is one of the most important capabilities that separates vector databases from simple ANN libraries like FAISS.

FAISS (Facebook AI Similarity Search) is an open-source ANN library from Meta — the fastest option for GPU-accelerated search. It is not a full database: it has no persistence, no metadata, and no built-in serving.

FAISS is ideal when you need maximum throughput on a single machine and manage persistence yourself. Chroma, Weaviate, and pgvector all use FAISS or HNSW under the hood.

Traditional SQL databases like PostgreSQL are optimized for exact lookups and range queries on structured data. They are not designed for high-dimensional nearest neighbor search. Even with the pgvector extension, pure PostgreSQL is slower than purpose-built vector databases for large corpora.

However, pgvector is an excellent choice when your application already runs on PostgreSQL and your corpus is under a few million documents, since it avoids adding another infrastructure component.

Vector database choices fall into two categories:

• Managed (serverless): Pinecone, Weaviate Cloud — no infrastructure to manage, pay per query/storage, immediate scalability
• Self-hosted: Qdrant, Chroma, Weaviate open-source, pgvector — full control, lower cost at scale, but you manage backups, upgrades, and scaling

For early-stage projects, start with a managed service to move quickly. Evaluate self-hosting when monthly costs exceed $200-300.

Different index types offer different trade-offs:

• Flat: Exact search, no approximation, slow at scale but zero accuracy loss — good for baseline benchmarking
• IVF (Inverted File): Partitions vectors into clusters, searches only the nearest clusters — fast but needs tuning of nlist and nprobe
• HNSW: Graph-based, best recall-speed trade-off for most workloads, the default in most production databases

Vector quantization compresses each 32-bit float in a vector to fewer bits, dramatically reducing memory usage at the cost of slight accuracy loss:

• FP32: 1536 dims × 4 bytes = 6KB per vector
• FP16: 3KB per vector — 2x compression, negligible accuracy loss
• INT8: 1.5KB per vector — 4x compression, ~1% recall drop

At 10 million vectors, INT8 quantization reduces memory from 60GB to 15GB, making the difference between fitting in RAM or not.

Consider switching from in-memory NumPy search to a vector database when:

• Your corpus exceeds 50,000 documents and query latency degrades
• You need metadata filtering (by date, user, category, etc.)
• You need persistence that survives application restarts
• Multiple services or users need to share the same index
• You need to update or delete individual documents without re-indexing everything

Test your understanding of AI Engineering concepts from this lesson.

In this lesson you learned: brute-force NumPy search does not scale beyond tens of thousands of documents, HNSW enables fast approximate nearest neighbor search by navigating a hierarchical graph, and vector databases add metadata filtering, persistence, and CRUD operations on top of ANN indexes. Next up we set up Pinecone, the most popular managed vector database, and index our first documents.