속도 기반 부하 차단 및 백프레셔

Circuit breakers and retries protect against dependency failures. But what protects a service from too much traffic of its own? Without limits, an overloaded service slows, queues grow, and it eventually falls over for everyone.

The goal under overload is graceful degradation: serve as many requests as you can well, and reject the rest fast, rather than serving everyone slowly until total failure.

Load shedding means dropping requests when the system is saturated. A quick 503 Service Unavailable is far better than a request that hangs and consumes resources.

Not all traffic is equal. Shed low-value load first:

• Keep paid or critical requests
• Drop background or best-effort work

This preserves the experience that matters most.

Backpressure is signaling upstream to slow down. Instead of silently buffering more than you can handle, you push the limit back toward the producer.

An unbounded queue hides overload until memory runs out. A bounded queue rejects new work when full, turning a slow death into an immediate, recoverable signal.

Cap the number of requests processed at once. Adaptive limiters adjust this cap based on observed latency — when responses slow, the limit shrinks automatically.

Reactive and streaming protocols build backpressure in: the consumer requests N items, and the producer sends no more than that until asked. Demand flows backward to match capacity.

Under overload, a quick rejection lets the client retry elsewhere or back off. A slow success ties up resources and cascades the slowdown to every other caller. Fail fast.

Resilient services layer them: concurrency limits bound work, bounded queues absorb short bursts, load shedding drops excess, and backpressure tells upstream to ease off.

Pair shedding with smart routing: a load balancer that reads each instance's health can stop sending traffic to a saturated node. The overloaded instance signals not ready, and traffic flows to peers with spare capacity.

You learned to survive overload:

• Load shedding drops excess, prioritizing critical traffic
• Backpressure signals upstream to slow down
• Bounded queues and concurrency limits cap work
• Fail fast rather than degrade everyone