การค้นหาแบบผสมด้วยเวกเตอร์เบาบางและหนาแน่น

Dense vectors capture meaning but can miss exact terms like product codes, names, or rare keywords. A user searching 'error E4012' may get semantically related but wrong results.

Hybrid search fixes this by adding keyword matching.

Two vector kinds:

• Dense — a few hundred floats encoding semantic meaning
• Sparse — mostly zeros, with weights only for present terms (like keyword scores)

Sparse vectors behave like classic keyword search.

A sparse vector is stored as indices and values for the non-zero terms.

Hybrid search combines the strengths:

• Dense handles synonyms and intent
• Sparse guarantees exact-term matches
• Together they boost recall and precision

To use hybrid search in Pinecone, create a dotproduct index and upsert each record with both a dense values array and a sparse_values field. Queries supply both representations of the query.

A record carries dense and sparse parts together.

Hybrid queries use an alpha parameter to weight dense vs sparse. alpha=1 is pure dense, alpha=0 is pure sparse. Tune it for your data.

Sparse vectors come from keyword models like BM25 or learned sparse encoders (e.g. SPLADE). They map terms to weighted indices that Pinecone can match against stored records.

The right alpha depends on your queries:

• Keyword-heavy domains (codes, IDs) -> lower alpha
• Natural-language questions -> higher alpha

Test on real queries and measure both recall and precision.

Reach for hybrid when exact terms matter: legal, medical, technical docs, or catalogs with SKUs. For purely conversational content, dense alone may be enough and simpler.

Hybrid search in Pinecone = a dotproduct index, records with dense and sparse values, queries supplying both, and a tuned alpha. It captures meaning and exact terms in one ranked result set.

You learned that hybrid search combines dense semantic vectors with sparse keyword vectors so Pinecone captures both meaning and exact terms. Use a dotproduct index, upsert both representations, and tune the alpha weighting to your query mix.