Vector Databases: Pinecone, Weaviate & pgvector · Lesson

Real-time Updates & Deletions

Learn techniques for handling dynamic data, including efficient real-time updates and deletions of vectors.

Real-time Updates & Deletions is a free Vector Databases: Pinecone, Weaviate & pgvector lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Vector Databases: Pinecone, Weaviate & pgvector learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

Handling Dynamic Vector Data

In real-world applications, data isn't static. It constantly changes! This means the vectors representing your data also need to be updated or removed from your vector database.

Imagine a product catalog where prices change, or user profiles where preferences evolve. Your vector database must reflect these changes to remain accurate.

Why Real-time Changes Matter

Real-time updates and deletions are crucial for maintaining the integrity and relevance of your applications:

Accuracy: Ensuring your search results are always based on the freshest data.
Relevance: Keeping recommendations and contextual information up-to-date for users.
Data Governance: Complying with data retention policies or user requests to delete personal data (e.g., 'right to be forgotten').

Full Vector & Metadata Updates

To completely replace an existing vector's embedding values and its associated metadata in Pinecone, you use the upsert() method.

If a vector with the same ID already exists in your index, upsert() will overwrite its existing vector values and metadata with the new ones you provide. If no vector with that ID exists, it will be inserted as a new entry.

Example: Replacing a Vector

If a product's description changes significantly, its embedding might need a full replacement. We use upsert() with the existing ID to update both the vector and its metadata.

from pinecone import Pinecone, Index
import os

# For this example, we'll use a mock class instead of actual Pinecone connection
# In a real app, replace with: 
# pc = Pinecone(api_key=os.environ.get("PINECONE_API_KEY"), environment=os.environ.get("PINECONE_ENVIRONMENT"))
# index = pc.Index("your-index-name")

class MockPineconeIndex:
    def upsert(self, vectors):
        print(f"Upserting vectors: {vectors}")

index = MockPineconeIndex() 

# Assume 'product-123' already exists with an old vector and metadata
print("--- Before Update (Conceptual) ---")
print("Vector 'product-123' has old values and metadata.")

# New vector values and metadata for 'product-123'
new_vector_values = [0.9, 0.8, 0.7, 0.6, 0.5, 0.4, 0.3, 0.2] # Example 8-dim vector
new_metadata = {"category": "electronics", "price": 129.99, "status": "updated"}

# Use upsert to fully replace the vector and metadata for 'product-123'
index.upsert(
    vectors=[
        {"id": "product-123", "values": new_vector_values, "metadata": new_metadata}
    ]
)

print("\n--- After Update (Conceptual) ---")
print("Vector 'product-123' now has new values and metadata, replacing old ones.")

Partial Updates with `update()`

Sometimes you only need to change a vector's metadata, or perhaps just its vector values without touching metadata. Pinecone provides an update() method for these partial modifications.

Using update() is more efficient than upsert() when you only have partial changes, as you only send the data that has changed, rather than replacing the entire entry.

Example: Updating Metadata Only

To update only the metadata for an existing vector, you use the update() method. You provide the vector's ID and the set_metadata parameter with the new metadata. The existing vector values will remain untouched.

from pinecone import Pinecone, Index
import os

# Mock class for Pinecone's update method
class MockPineconeIndexUpdate:
    def update(self, id, values=None, set_metadata=None):
        print(f"Updating vector ID: {id}")
        if values: print(f"  New values provided (vector will be replaced): {values}")
        if set_metadata: print(f"  Setting metadata: {set_metadata}")

index_update = MockPineconeIndexUpdate() # Replace with pc.Index("your-index-name")

# Assume 'product-456' exists with vector and metadata like {'stock': 10, 'color': 'blue'}
print("--- Before Metadata Update (Conceptual) ---")
print("Vector 'product-456' has existing metadata.")

# Update only the stock count for 'product-456'
updated_metadata = {"stock": 5, "last_checked": "2023-10-27"}

index_update.update(id="product-456", set_metadata=updated_metadata)

print("\n--- After Metadata Update (Conceptual) ---")
print("Vector 'product-456' now has updated stock and last_checked metadata. Other metadata and vector values are unchanged.")

Deleting Vectors in Pinecone

Removing vectors from your Pinecone index is a common operation. Pinecone offers flexible ways to delete vectors:

By one or more specific ID(s).
By matching a metadata filter.
Deleting all vectors within an index or a specific namespace.

This functionality is vital for data hygiene, managing outdated information, and adhering to privacy regulations.

Example: Deleting a Single Vector

The most straightforward way to delete a vector is by its unique ID. You pass a list of one or more IDs to the delete() method.

from pinecone import Pinecone, Index
import os

# Mock class for Pinecone's delete method
class MockPineconeIndexDelete:
    def delete(self, ids=None, delete_all=False, filter=None):
        if ids: print(f"Deleting vectors with IDs: {ids}")
        elif delete_all: print("Deleting all vectors.")
        elif filter: print(f"Deleting vectors with filter: {filter}")

index_delete = MockPineconeIndexDelete() # Replace with pc.Index("your-index-name")

# Assume 'user-profile-789' exists in the index
print("--- Before Deletion (Conceptual) ---")
print("Vector 'user-profile-789' is present.")

# Delete a single vector by its ID
index_delete.delete(ids=["user-profile-789"])

print("\n--- After Deletion (Conceptual) ---")
print("Vector 'user-profile-789' has been removed from the index.")

Deleting Many Vectors by Filter

While you can delete multiple vectors by providing a list of IDs, a powerful feature is deleting vectors based on a metadata filter.

This allows you to efficiently remove all vectors that match specific metadata criteria, such as all products from a certain category, or all inactive user profiles, without knowing their individual IDs.

Example: Deleting with Filters

Let's say we want to remove all vectors associated with products marked as 'out_of_stock'. We can use a filter object within the delete() method.

from pinecone import Pinecone, Index
import os

# Mock class for Pinecone's delete method
class MockPineconeIndexDelete:
    def delete(self, ids=None, delete_all=False, filter=None):
        if ids: print(f"Deleting vectors with IDs: {ids}")
        elif delete_all: print("Deleting all vectors.")
        elif filter: print(f"Deleting vectors with filter: {filter}")

index_delete = MockPineconeIndexDelete() # Replace with pc.Index("your-index-name")

# Assume several products exist, some with 'status': 'out_of_stock'
print("--- Before Filtered Deletion (Conceptual) ---")
print("Vectors for products with status 'out_of_stock' are present.")

# Delete all vectors where the 'status' metadata is 'out_of_stock'
index_delete.delete(filter={
    "status": "out_of_stock"
})

print("\n--- After Filtered Deletion (Conceptual) ---")
print("All vectors for 'out_of_stock' products have been removed.")

Check Your Understanding

Which of the following statements about updating and deleting vectors in Pinecone are TRUE?

Recap & Next Steps

Great job! You've learned how to keep your Pinecone index dynamic and up-to-date.

We explored how upsert() performs a full replacement of vectors and metadata.
You saw how the update() method is used for efficient partial changes, especially for metadata.
Finally, we covered various ways to delete vectors, including by ID and using powerful metadata filters.

These operations are essential for managing evolving datasets in real-world AI applications.

Frequently asked questions

Is the “Real-time Updates & Deletions” lesson free?

Yes — the full text of “Real-time Updates & Deletions” is free to read here on the web, and the Vector Databases: Pinecone, Weaviate & pgvector course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Vector Databases: Pinecone, Weaviate & pgvector course, upgrade to CoddyKit PRO.

What will I learn in “Real-time Updates & Deletions”?

Learn techniques for handling dynamic data, including efficient real-time updates and deletions of vectors. You practise Vector Databases: Pinecone, Weaviate & pgvector with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start Vector Databases: Pinecone, Weaviate & pgvector?

No prior experience is required. Vector Databases: Pinecone, Weaviate & pgvector on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “Real-time Updates & Deletions” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this Vector Databases: Pinecone, Weaviate & pgvector lesson?

Yes. Every Vector Databases: Pinecone, Weaviate & pgvector lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

← Back to Vector Databases: Pinecone, Weaviate & pgvector