Circuit Breakers and Graceful Degradation
Learn how circuit breakers prevent cascading failures and how graceful degradation keeps a system useful even when dependencies fail.
Circuit Breakers and Graceful Degradation is a free System Design Basics for Backend Developers lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the System Design Basics for Backend Developers learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
The Cascading Failure Problem
In a system of dependent services, one slow service can drag down everything that calls it. Threads pile up waiting, queues fill, and the failure cascades across the whole system.
High availability means containing failures, not just preventing them.
What a Circuit Breaker Does
A circuit breaker wraps calls to a dependency. When failures cross a threshold, it opens and fails fast instead of waiting on a dead service.
- Stops wasting threads on doomed calls
- Gives the failing service time to recover
The Three States
A circuit breaker has three states:
- Closed: calls flow normally, failures are counted
- Open: calls fail immediately without hitting the dependency
- Half-open: a few trial calls test whether the dependency recovered
CLOSED --(too many failures)--> OPEN
OPEN --(timeout elapsed)--> HALF_OPEN
HALF_OPEN --(trial succeeds)--> CLOSED
HALF_OPEN --(trial fails)--> OPENA Simple Breaker in Code
Here is the core idea: count failures, trip when a threshold is reached, and refuse calls while open.
class Breaker:
def __init__(self, limit):
self.fails = 0
self.limit = limit
self.open = False
def call(self, ok):
if self.open:
return 'rejected'
if ok:
self.fails = 0
return 'success'
self.fails += 1
if self.fails >= self.limit:
self.open = True
return 'failure'
b = Breaker(3)
for ok in [False, False, False, True]:
print(b.call(ok))Timeouts Are Essential
A breaker only helps if calls have timeouts. Without a timeout, a hung dependency holds a thread forever and failures are never counted. Always set aggressive, explicit timeouts on remote calls.
Retries and Backoff
Retries can help with transient errors but can also amplify an overload. Use exponential backoff with jitter and cap the retry count. Combine with a circuit breaker so retries stop entirely when the circuit is open.
import random
delay = 1
for attempt in range(4):
wait = delay + random.uniform(0, delay)
print('attempt', attempt, 'wait', round(wait, 2))
delay *= 2Graceful Degradation
Graceful degradation means the system still does something useful when a dependency is down, instead of returning an error.
- Serve stale cached data
- Hide a non-critical feature
- Return a sensible default
Fallbacks
When the breaker is open, route to a fallback. For a product page, if the recommendations service is down, show a generic best-sellers list instead of failing the whole page.
def get_recommendations(breaker):
if breaker.open:
return ['bestseller-1', 'bestseller-2']
return ['personalized-1', 'personalized-2']Bulkheads
The bulkhead pattern isolates resources so one failing dependency cannot consume all threads or connections. Give each downstream dependency its own bounded pool — like watertight compartments in a ship.
Load Shedding
Under extreme load, it is better to reject some requests quickly than to slow down for everyone. Load shedding drops low-priority traffic to protect critical paths and keep latency bounded.
Putting It Together
Resilient services layer these patterns: tight timeouts, circuit breakers, bulkheads to isolate, fallbacks for degradation, and load shedding under pressure. Together they turn a potential outage into a minor, contained blip.
Quick Check
Test your understanding of circuit breakers.
Recap
You learned to contain failures for high availability:
- Circuit breakers fail fast and cycle through closed, open, and half-open
- Timeouts and capped backoff retries prevent overload amplification
- Graceful degradation and fallbacks keep the system useful
- Bulkheads and load shedding isolate and protect critical paths
Frequently asked questions
Is the “Circuit Breakers and Graceful Degradation” lesson free?
Yes — the full text of “Circuit Breakers and Graceful Degradation” is free to read here on the web, and the System Design Basics for Backend Developers course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the System Design Basics for Backend Developers course, upgrade to CoddyKit PRO.
What will I learn in “Circuit Breakers and Graceful Degradation”?
Learn how circuit breakers prevent cascading failures and how graceful degradation keeps a system useful even when dependencies fail. You practise System Design Basics for Backend Developers with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start System Design Basics for Backend Developers?
No prior experience is required. System Design Basics for Backend Developers on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Circuit Breakers and Graceful Degradation” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this System Design Basics for Backend Developers lesson?
Yes. Every System Design Basics for Backend Developers lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Redundancy and Failover Mechanisms
- Disaster Recovery Planning
- Monitoring, Alerting, and Logging
- Circuit Breakers and Graceful Degradation