System Design Basics for Backend Developers · Lesson

Cache Eviction Policies

Explore how caches decide what to keep and what to discard when memory is full, covering LRU, LFU, FIFO, TTL, and their trade-offs.

Cache Eviction Policies is a free System Design Basics for Backend Developers lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the System Design Basics for Backend Developers learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

Why Eviction Is Necessary

A cache is fast because it lives in limited memory. When it fills up, it must evict something to make room for new data. The eviction policy decides what to drop.

A good policy keeps the data most likely to be reused.

Hit Rate Is the Goal

The metric that matters is the hit rate: the fraction of requests served from cache. A better eviction policy raises the hit rate, which means fewer slow trips to the database or origin.

hits = 850
misses = 150
hit_rate = hits / (hits + misses)
print('hit rate:', hit_rate)

FIFO

FIFO (First In, First Out) evicts the oldest inserted item regardless of usage. It is simple but ignores access patterns, so a frequently used old item can be wrongly evicted.

LRU: Least Recently Used

LRU evicts the item that has not been accessed for the longest time. It assumes recently used data will be used again soon — true for most workloads, which is why LRU is the default in many caches.

Implementing LRU

A classic LRU uses an ordered map. On access, move the key to the most-recent end; when full, evict from the least-recent end.

from collections import OrderedDict
class LRU:
    def __init__(self, cap):
        self.cap = cap
        self.d = OrderedDict()
    def get(self, k):
        if k in self.d:
            self.d.move_to_end(k)
            return self.d[k]
    def put(self, k, v):
        self.d[k] = v
        self.d.move_to_end(k)
        if len(self.d) > self.cap:
            self.d.popitem(last=False)

c = LRU(2)
c.put('a', 1); c.put('b', 2); c.get('a'); c.put('c', 3)
print(list(c.d.keys()))

LFU: Least Frequently Used

LFU evicts the item accessed the fewest times. It favors long-term popular items over recent bursts. The downside: a once-popular item can linger long after it stops being useful.

TTL-Based Expiration

A TTL (time to live) expires items after a fixed duration regardless of memory pressure. It bounds staleness and is often combined with LRU: TTL controls freshness, LRU controls memory.

SET session:42 "..." EX 3600
# expires in 3600 seconds

Random and Allkeys Variants

Some caches offer random eviction (cheap, surprisingly decent) and scope variants: evict only keys with a TTL set, or evict from all keys. Redis exposes policies like allkeys-lru and volatile-ttl.

Thundering Herd on Eviction

When a hot key is evicted or expires, many clients may simultaneously rebuild it — a thundering herd that hammers the origin. Mitigate with request coalescing (single-flight) or slightly randomized TTLs to spread out expirations.

Choosing a Policy

Match the policy to the workload:

Recency-driven traffic: LRU
Stable popularity: LFU
Freshness-critical data: TTL
Uniform access: random is fine and cheap

Eviction in a CDN

CDNs apply the same ideas at the edge. Each edge node has finite storage and evicts (often LRU) plus honors Cache-Control: max-age as a TTL. Understanding eviction explains why a cold edge node has a low initial hit rate.

Quick Check

Test your understanding of eviction policies.

Recap

You learned how caches manage limited memory:

Eviction policies aim to maximize hit rate
FIFO, LRU, LFU, TTL, and random each suit different patterns
Thundering herds need coalescing or jittered TTLs
CDNs apply the same eviction logic at the edge

Frequently asked questions

Is the “Cache Eviction Policies” lesson free?

Yes — the full text of “Cache Eviction Policies” is free to read here on the web, and the System Design Basics for Backend Developers course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the System Design Basics for Backend Developers course, upgrade to CoddyKit PRO.

What will I learn in “Cache Eviction Policies”?

Explore how caches decide what to keep and what to discard when memory is full, covering LRU, LFU, FIFO, TTL, and their trade-offs. You practise System Design Basics for Backend Developers with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start System Design Basics for Backend Developers?

No prior experience is required. System Design Basics for Backend Developers on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “Cache Eviction Policies” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this System Design Basics for Backend Developers lesson?

Yes. Every System Design Basics for Backend Developers lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

← Back to System Design Basics for Backend Developers