0Pricing
AI Agents · Lesson

Caching Prompts and Results (Anthropic, Vertex)

Anthropic prompt caching and Vertex caching cut input cost by 10x on long, repeated system prompts.

Caching Prompts and Results (Anthropic, Vertex) is a free AI Agents lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Agents learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

Why Cache?

Many agent calls are nearly identical: same system prompt, same few-shot examples, different user input. Without caching, you re-process the static parts every call.

Caching can cut input token cost by 10x.

Two Kinds of Caching

  1. Prompt caching (server-side) — provider caches the model's KV state for repeated prefixes
  2. Result caching (client-side) — your code caches full responses for identical inputs

Anthropic Prompt Caching

Mark cacheable parts with cache_control:

response = client.messages.create(
    model='claude-sonnet-4-5',
    max_tokens=1024,
    system=[
        {'type': 'text', 'text': LONG_SYSTEM_PROMPT, 'cache_control': {'type': 'ephemeral'}}
    ],
    messages=[{'role': 'user', 'content': user_input}]
)
# 1st call: full price
# 2nd call within 5min: 10% of input cost

Cache Hit Ratio

Check the response usage object:

response.usage.cache_creation_input_tokens   # tokens cached this call
response.usage.cache_read_input_tokens       # tokens read from cache

What to Cache

  • System prompts > 1k tokens
  • Tool definitions
  • Few-shot examples
  • RAG context shared across queries (rare)

Where to Place Cache Markers

Cache control must be on the LAST static block. Anything before the marker is cached. Stable content at the start; variable content at the end.

OpenAI Prompt Caching

OpenAI auto-caches prompts > 1024 tokens. No explicit marker:

# Just send the same prefix repeatedly. Cache discount applies automatically.
# Check response.usage.prompt_tokens_details.cached_tokens

Vertex AI Context Caching

Google Vertex requires you to explicitly create a cache object:

from vertexai.generative_models import GenerativeModel, content_cache

cache = content_cache.CachedContent.create(
    model='gemini-1.5-pro',
    contents=[long_system_content],
    ttl=300
)
model = GenerativeModel.from_cached_content(cache)
response = model.generate_content(user_input)

Client-Side Result Cache

For exact-match queries (same input, same expected output), use a key-value cache:

import hashlib

def cache_key(model, messages):
    return hashlib.sha256(f'{model}:{json.dumps(messages)}'.encode()).hexdigest()

def cached_call(model, messages):
    key = cache_key(model, messages)
    if cached := redis.get(key):
        return json.loads(cached)
    result = real_call(model, messages)
    redis.set(key, json.dumps(result), ex=3600)
    return result

Semantic Cache

Use embeddings to cache similar (not identical) queries:

qvec = embed(query)
matches = vector_db.query(qvec, k=1)
if matches and matches[0].score > 0.95:
    return matches[0].metadata['cached_response']

Cache Invalidation

Stale caches return wrong answers. Strategies:

  • TTL — short for time-sensitive data
  • Manual invalidation on data updates
  • Per-user keys so updates only affect one user

Don't Cache Personalised Responses

"What is my balance?" cannot be cached across users. Be careful with shared caches in multi-tenant systems.

Cache Cost vs LLM Cost

Redis costs are negligible vs LLM costs. Cache aggressively — even 10% hit rate saves real money.

Real-World Savings

With long shared system prompts and prompt caching, teams routinely see input cost drop 50-90% in production. One of the highest-ROI optimisations.

Anthropic Cache Trigger

How do you enable prompt caching with Anthropic?

Recap

Server-side prompt caching for static prefixes; client-side KV cache for exact-match; semantic cache for fuzzy match. Always check hit ratios and savings.

Frequently asked questions

Is the “Caching Prompts and Results (Anthropic, Vertex)” lesson free?

Yes — the full text of “Caching Prompts and Results (Anthropic, Vertex)” is free to read here on the web, and the AI Agents course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Agents course, upgrade to CoddyKit PRO.

What will I learn in “Caching Prompts and Results (Anthropic, Vertex)”?

Anthropic prompt caching and Vertex caching cut input cost by 10x on long, repeated system prompts. You practise AI Agents with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start AI Agents?

No prior experience is required. AI Agents on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “Caching Prompts and Results (Anthropic, Vertex)” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this AI Agents lesson?

Yes. Every AI Agents lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

  1. Token Budgets Per Step
  2. Model Routing (Cheap -> Expensive)
  3. Caching Prompts and Results (Anthropic, Vertex)
  4. Quantisation and Speculative Decoding
← Back to AI Agents