AI Engineering Academy · Leçon

Mise en cache exacte avec Redis

Mise en cache exacte avec Redis est une leçon AI Engineering Academy gratuite sur CoddyKit. Ceci est la leçon 1 sur 4. Tu peux lire la leçon complète ci-dessous gratuitement — puis la pratiquer en direct dans le navigateur avec un éditeur de code intégré et un tuteur IA 24/7. Elle fait partie du parcours d'apprentissage AI Engineering Academy, et ta progression se synchronise sur le web et l'application CoddyKit. Le cours AI Engineering Academy comprend 4 leçons au total.

Certaines parties de cette leçon n'ont pas encore été traduites et s'affichent en anglais.

Why Cache LLM Responses?

LLM API calls are expensive: a single GPT-4o request can cost $0.005-$0.15 depending on token count. In many applications, a significant fraction of incoming queries are identical or near-identical to previous ones — think FAQ bots, customer support systems, or code review tools where users ask the same questions repeatedly. Caching can eliminate 20-50 percent of API calls in these use cases, directly cutting costs and reducing latency.

Exact Cache: Cache Key Design

An exact cache stores LLM responses keyed by a deterministic hash of the input. The cache key must capture every input that affects the output: the messages array, the model name, temperature, and any other parameters that change the response. Missing any of these from the key causes cache collisions where a cached response is served for a different effective request.

import hashlib
import json

def make_cache_key(messages: list[dict], model: str, temperature: float) -> str:
    # Create a canonical, order-stable representation
    key_data = {
        'model': model,
        'temperature': temperature,
        'messages': messages,  # list order matters
    }
    # Serialize to JSON with sorted keys for determinism
    serialized = json.dumps(key_data, sort_keys=True, ensure_ascii=False)
    # Hash to a fixed-length key safe for Redis
    return 'llm_cache:' + hashlib.sha256(serialized.encode()).hexdigest()

Connecting to Redis

Redis is the standard choice for LLM response caching due to its sub-millisecond read latency and built-in TTL support. Use the redis-py library for synchronous access or aioredis (now merged into redis-py as redis.asyncio) for async access in FastAPI applications. Store the Redis connection as a singleton to avoid connection pool exhaustion.

import redis
import redis.asyncio as aioredis

# Synchronous Redis client
r = redis.Redis(
    host='localhost',
    port=6379,
    db=0,
    decode_responses=True,  # return str instead of bytes
)

# Async Redis client (for FastAPI)
async_r = aioredis.Redis(
    host='localhost',
    port=6379,
    db=0,
    decode_responses=True,
)

# Test connection
print(r.ping())  # True if Redis is running

Cache-Aside Pattern Implementation

The cache-aside pattern is the standard caching strategy for LLM APIs. On each request: (1) compute the cache key, (2) check Redis for a cached response, (3) if found (cache hit) return it immediately, (4) if not found (cache miss) call the LLM API, (5) store the response in Redis with a TTL, (6) return the response. This pattern keeps caching logic external to the LLM call itself.

import json
from openai import OpenAI

client = OpenAI()

def cached_completion(
    messages: list[dict],
    model: str = 'gpt-4o-mini',
    temperature: float = 0.7,
    ttl_seconds: int = 3600,
) -> str:
    cache_key = make_cache_key(messages, model, temperature)

    # Cache hit?
    cached = r.get(cache_key)
    if cached is not None:
        print('[CACHE HIT]')
        return json.loads(cached)

    # Cache miss: call API
    print('[CACHE MISS]')
    response = client.chat.completions.create(
        model=model,
        messages=messages,
        temperature=temperature,
    )
    result = response.choices[0].message.content

    # Store in cache with TTL
    r.setex(cache_key, ttl_seconds, json.dumps(result))
    return result

Async Cache-Aside for FastAPI

In an async FastAPI application, use the async Redis client so cache lookups do not block the event loop. The pattern is identical to the synchronous version but uses await for all Redis operations. This keeps the caching layer fully non-blocking and compatible with the async LLM client.

from openai import AsyncOpenAI
import redis.asyncio as aioredis
import json

async_client = AsyncOpenAI()
async_r = aioredis.Redis(host='localhost', port=6379, decode_responses=True)

async def async_cached_completion(
    messages: list[dict],
    model: str = 'gpt-4o-mini',
    temperature: float = 0.0,
    ttl: int = 86400,
) -> str:
    key = make_cache_key(messages, model, temperature)

    cached = await async_r.get(key)
    if cached:
        return json.loads(cached)

    response = await async_client.chat.completions.create(
        model=model, messages=messages, temperature=temperature
    )
    result = response.choices[0].message.content
    await async_r.setex(key, ttl, json.dumps(result))
    return result

Choosing the Right TTL

The TTL (Time To Live) controls how long cached responses remain valid. For factual Q&A with stable knowledge bases, long TTLs (24-72 hours) maximize cache hit rates. For responses that should reflect the latest data (news summarization, live prices), short TTLs (5-15 minutes) or no caching at all are appropriate. For creative tasks with non-zero temperature, caching may produce stale responses — consider caching only for temperature=0.

# TTL strategy by use case
TTL_STRATEGY = {
    'faq_answering':          86400 * 7,   # 7 days — stable facts
    'code_explanation':       86400,        # 1 day — code rarely changes
    'document_summarization': 3600 * 6,    # 6 hours
    'news_analysis':          300,          # 5 minutes — stale quickly
    'creative_writing':       0,            # 0 = don't cache (non-deterministic)
}

def get_ttl_for_use_case(use_case: str) -> int:
    return TTL_STRATEGY.get(use_case, 3600)  # default 1 hour

Caching Metrics and Monitoring

Track cache hit rate as a primary cost-reduction metric. A cache hit rate of 30 percent means 30 percent of API calls are avoided. Store hit and miss counts in Redis itself using INCR commands on separate counters. Expose a /metrics endpoint in your FastAPI app that reports current hit rate, total requests, and estimated cost savings to quantify the ROI of caching.

CACHE_HITS_KEY = 'llm_cache_metrics:hits'
CACHE_MISSES_KEY = 'llm_cache_metrics:misses'

async def async_cached_completion_instrumented(messages, model, temperature=0.0):
    key = make_cache_key(messages, model, temperature)
    cached = await async_r.get(key)

    if cached:
        await async_r.incr(CACHE_HITS_KEY)
        return json.loads(cached)

    await async_r.incr(CACHE_MISSES_KEY)
    response = await async_client.chat.completions.create(
        model=model, messages=messages, temperature=temperature
    )
    result = response.choices[0].message.content
    await async_r.setex(key, 3600, json.dumps(result))
    return result

async def get_cache_stats():
    hits = int(await async_r.get(CACHE_HITS_KEY) or 0)
    misses = int(await async_r.get(CACHE_MISSES_KEY) or 0)
    total = hits + misses
    return {'hit_rate': hits / total if total > 0 else 0, 'total': total}

Cache Invalidation Strategies

Exact cache invalidation is straightforward because keys are deterministic. To invalidate a specific entry, recompute its key and call r.delete(key). To invalidate all entries for a specific prompt pattern, use Redis key prefixes with a wildcard scan. To invalidate the entire cache on a major knowledge base update, call r.flushdb() (use with caution — this deletes all keys in the database).

async def invalidate_cache_entry(messages, model, temperature):
    key = make_cache_key(messages, model, temperature)
    deleted = await async_r.delete(key)
    print(f'Deleted {deleted} cache entries')

async def invalidate_all_llm_cache():
    # Scan for all keys with prefix 'llm_cache:'
    keys_to_delete = []
    async for key in async_r.scan_iter(match='llm_cache:*'):
        keys_to_delete.append(key)
    if keys_to_delete:
        await async_r.delete(*keys_to_delete)
    print(f'Invalidated {len(keys_to_delete)} cache entries')

Serializing Complex Responses

If your application caches entire API response objects (not just the text content), serialize them carefully. The full ChatCompletion object includes token usage, model version, and finish reason — useful for logging and cost tracking. Use the SDK's .model_dump_json() method to serialize Pydantic response objects to JSON strings, and reconstruct them with ChatCompletion.model_validate_json() on cache retrieval.

from openai.types.chat import ChatCompletion

async def cached_completion_full_response(
    messages, model='gpt-4o-mini', temperature=0.0
):
    key = make_cache_key(messages, model, temperature) + ':full'
    cached = await async_r.get(key)

    if cached:
        return ChatCompletion.model_validate_json(cached)  # reconstruct object

    response = await async_client.chat.completions.create(
        model=model, messages=messages, temperature=temperature
    )
    # Serialize Pydantic model to JSON
    await async_r.setex(key, 3600, response.model_dump_json())
    return response

Caching and Non-Determinism

Exact caching only makes sense for deterministic or near-deterministic requests. At temperature=0 and top_p=1.0, most LLMs produce the same output for the same input (though not guaranteed due to floating-point non-determinism). At higher temperatures, cached responses become stale as the model would have produced different outputs. Always cache at temperature=0 or document clearly in your cache key that responses may vary.

Redis Cluster and Production Setup

For production deployments with high cache volumes, use Redis Cluster for horizontal sharding across multiple nodes, or a managed Redis service like AWS ElastiCache or Redis Cloud. Set a maxmemory policy (typically allkeys-lru to evict the least recently used entries when memory is full) to prevent Redis from running out of memory and automatically manage cache size.

# Redis configuration for production LLM caching
# In redis.conf:
# maxmemory 2gb
# maxmemory-policy allkeys-lru

# Connection with retry and connection pool
import redis
from redis.retry import Retry
from redis.backoff import ExponentialBackoff

retry = Retry(ExponentialBackoff(base=0.1), 3)
production_redis = redis.Redis(
    host='your-redis-host.cache.amazonaws.com',
    port=6379,
    ssl=True,
    decode_responses=True,
    max_connections=50,
    retry=retry,
    retry_on_error=[redis.ConnectionError, redis.TimeoutError],
)

Quick Check

Test your understanding of exact LLM response caching with Redis from this lesson.

Lesson Recap

In this lesson you learned: exact caching hashes all LLM inputs to produce a deterministic cache key, the cache-aside pattern checks Redis before calling the API and stores results after a miss, and TTL selection should reflect how frequently your content changes — longer for stable knowledge, shorter for dynamic data. Monitor cache hit rate as a primary cost-reduction metric. Next up we build semantic caching for similar but non-identical queries.

Questions Fréquemment Posées

La leçon « Mise en cache exacte avec Redis » est-elle gratuite ?

Oui — le texte complet de « Mise en cache exacte avec Redis » est gratuit à lire ici sur le web. Pour la pratiquer de manière interactive (un éditeur de code intégré et un tuteur IA 24/7) et déverrouiller le reste du cours AI Engineering Academy, passe à CoddyKit PRO. Le cours AI Engineering Academy comprend 4 leçons au total.

Qu'est-ce que j'apprendrai dans « Mise en cache exacte avec Redis » ?

Mettez en cache les réponses du LLM en hachant l’invite complète et en stockant le résultat dans Redis avec un TTL, afin de servir instantanément les requêtes identiques sans effectuer d’appel à l’AP… Tu pratiques AI Engineering Academy avec du code pratique que tu exécutes directement dans le navigateur, et un tuteur IA 24/7 répond à tes questions au fur et à mesure que tu avances dans la leçon.

Dois-je avoir de l'expérience pour commencer AI Engineering Academy ?

Aucune expérience préalable n'est requise. AI Engineering Academy sur CoddyKit est structuré pour les débutants jusqu'aux apprenants avancés, donc tu peux commencer ici ou depuis le début et avancer à ton rythme. Ceci est la leçon 1 sur 4.

Combien de temps prend la leçon « Mise en cache exacte avec Redis » ?

La plupart des leçons CoddyKit prennent environ 5–10 minutes. Chacune est courte et interactive, tu progresses régulièrement et tu repiques exactement où tu t'es arrêté sur le web et l'app.

Peux-tu écrire et exécuter du code dans cette leçon AI Engineering Academy ?

Oui. Chaque leçon AI Engineering Academy inclut un éditeur de code intégré, tu écris et exécutes du vrai code directement dans ton navigateur et tu reçois des retours IA instantanés — aucune configuration locale requise.

Toutes les leçons de ce cours

Mise en cache exacte avec Redis
Mise en cache sémantique avec des plongements
Mise en cache des préfixes d’invites d’OpenAI
Traitement par lots, routage des modèles et tableaux de bord des coûts

← Retour à AI Engineering Academy