Alerting and SLOs
Learn how to turn observability data into actionable alerts using Service Level Objectives, error budgets, and symptom-based alerting that reduces noise and pages humans only when it matters.
Alerting and SLOs is a free Microservices Communication Patterns (Saga, Circuit Breaker) lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Microservices Communication Patterns (Saga, Circuit Breaker) learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
From Observability to Action
Traces, logs, and metrics tell you what is happening. Alerting decides when a human needs to act. The goal is to page on real user pain, not on every blip.
What Is an SLI?
A Service Level Indicator is a measured aspect of service behavior, such as request latency, error rate, or availability. SLIs are the raw signal you alert on.
ok = 995
total = 1000
availability = ok / total * 100
print('Availability SLI:', availability, '%')What Is an SLO?
A Service Level Objective is a target for an SLI over a window, for example 'availability >= 99.9% over 30 days'. It defines what 'good enough' means.
Error Budgets
If your SLO is 99.9%, then 0.1% of requests are allowed to fail. That 0.1% is your error budget. Spend it on risk; if you burn it, slow down and stabilize.
slo = 99.9
budget_pct = 100 - slo
monthly_requests = 1000000
allowed_failures = monthly_requests * budget_pct / 100
print('Error budget (failures/month):', allowed_failures)Symptom vs Cause Alerts
Alert on symptoms users feel (high error rate, slow responses), not on every internal cause (one pod restarted). Cause alerts create noise; symptom alerts capture real impact.
Burn-Rate Alerting
Instead of paging the instant the SLO is missed, alert on how fast you are burning the error budget. A fast burn pages immediately; a slow burn opens a ticket.
def burn_rate(observed_error_rate, budget_rate):
return observed_error_rate / budget_rate
print('Burn rate:', burn_rate(0.01, 0.001), 'x budget')Reducing Alert Fatigue
Too many alerts and engineers ignore them all. Keep pages rare and meaningful:
- Every page must be actionable.
- Route non-urgent issues to tickets.
- Group related alerts together.
The Four Golden Signals
Google's SRE book recommends alerting on four signals:
- Latency
- Traffic
- Errors
- Saturation
Cover these and you catch most user-facing problems.
Severity Levels
Classify alerts by severity so the response matches the impact.
severity = {'P1': 'page on-call now', 'P2': 'page during hours', 'P3': 'create ticket'}
print(severity['P1'])Runbooks
Attach a runbook link to every alert. It tells the responder what the alert means, how to confirm impact, and the first steps to mitigate. Runbooks turn panic into procedure.
Closing the Loop
After each incident, review which alerts fired (and which should have). Tune thresholds, delete noisy alerts, and update runbooks. Alerting quality improves through iteration.
Quick Check
You have an SLO of 99.9% availability. What does the remaining 0.1% represent?
Recap
You learned alerting and SLOs:
- SLIs measure behavior; SLOs set targets; error budgets quantify allowed failure.
- Alert on symptoms and burn rate, not every cause.
- Use severity levels and runbooks to make pages actionable.
- Iterate to cut alert fatigue.
Good alerting pages humans only when users actually hurt.
Frequently asked questions
Is the “Alerting and SLOs” lesson free?
Yes — the full text of “Alerting and SLOs” is free to read here on the web, and the Microservices Communication Patterns (Saga, Circuit Breaker) course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Microservices Communication Patterns (Saga, Circuit Breaker) course, upgrade to CoddyKit PRO.
What will I learn in “Alerting and SLOs”?
Learn how to turn observability data into actionable alerts using Service Level Objectives, error budgets, and symptom-based alerting that reduces noise and pages humans only when it matters. You practise Microservices Communication Patterns (Saga, Circuit Breaker) with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Microservices Communication Patterns (Saga, Circuit Breaker)?
No prior experience is required. Microservices Communication Patterns (Saga, Circuit Breaker) on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Alerting and SLOs” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Microservices Communication Patterns (Saga, Circuit Breaker) lesson?
Yes. Every Microservices Communication Patterns (Saga, Circuit Breaker) lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.