영향 범위와 정상 상태 가설 측정

Chaos engineering is not random breakage; it is an experiment. Like any experiment, it needs a hypothesis, a controlled variable, and a measurable outcome.

This lesson focuses on the steady-state hypothesis and bounding the blast radius.

Steady state is your system's normal, healthy behavior expressed as measurable output, not internal metrics.

• Good: orders completed per minute, p99 latency
• Weak: CPU usage, memory

Steady state should reflect what users experience.

A chaos hypothesis predicts that steady state holds despite a specific fault. It must be falsifiable.

Blast radius is the maximum harm an experiment could cause: which users, services, and data could be affected if it goes wrong.

Controlling it is what separates a safe experiment from an outage you caused yourself.

Begin with the smallest meaningful scope and expand only after success.

• One instance before one zone
• 1% of traffic before 100%
• Staging before production

Define in advance when to stop. If steady state degrades past a threshold, the experiment must halt automatically.

Record the steady-state metric across three windows: a baseline before the fault, the period during it, and recovery after. Comparing these tells you whether the hypothesis held and how fast the system recovered.

Two outcomes are both valuable:

• Hypothesis holds: confidence that the system tolerates this fault
• Hypothesis fails: you found a weakness safely, before customers did

A failed hypothesis is a successful experiment.

Estimate worst-case impact numerically before running: affected users, potential revenue at risk, recovery time. This makes the go/no-go decision explicit rather than gut feeling.

Even a well-bounded experiment can surprise on-call. Announce the window, scope, and abort plan beforehand so a real incident is not confused with your test, and so help is ready if needed.

Putting it together:

• Define a user-facing steady-state metric
• State a falsifiable hypothesis
• Bound the blast radius and start small
• Set automatic abort conditions
• Measure before/during/after and learn from either result

Test your understanding of experiment design.

You learned to design safe, scientific chaos experiments.

• Define steady state from user-facing output
• Form falsifiable hypotheses
• Bound and quantify blast radius, start small
• Set abort conditions and learn from any result