Backpressure

Backpressure is a flow-control mechanism that prevents a fast producer from overwhelming a slow consumer. Instead of buffering unboundedly or dropping data, the consumer signals how much it can handle.

Akka Streams implements the Reactive Streams standard, where demand flows upstream and elements flow downstream.

Each stage only emits when the next stage has signalled demand. A Sink requests N elements; that demand propagates upstream until a Source produces exactly what was asked for.

This pull-based protocol means producers never push more than consumers can process.

Without backpressure, a fast Kafka consumer feeding a slow database would accumulate millions of in-flight records, exhausting memory and crashing the process.

Backpressure naturally throttles the upstream to the slowest stage, giving stable memory usage under load.

Between asynchronous boundaries, Akka Streams keeps a small internal buffer (default 16 elements). It absorbs short bursts so stages need not lock-step on every element.

When the buffer fills, backpressure kicks in and the upstream stops producing until space frees up.

The buffer operator inserts an explicit buffer of a chosen size with an OverflowStrategy that decides what happens when it is full.

This lets you trade memory for the ability to decouple producer and consumer speed.

Strategies include backpressure (slow the upstream), dropHead/dropTail (discard oldest or newest), dropBuffer, dropNew, and fail (terminate with an error).

Dropping strategies suit live data like sensor readings where stale values can be discarded safely.

When a consumer is slow, conflate merges pending elements into one using a combine function instead of buffering them all.

For example, collapse many numeric updates into their sum, so the consumer always sees an aggregate of what it missed.

expand is the dual of conflate: when downstream demands faster than upstream produces, it synthesizes extra elements from the last seen value.

This is useful to keep emitting a most-recent reading at a steady rate.

By default, fused stages run on a single actor with no buffering between them. Inserting async places a stage on its own actor, adding a buffer and enabling pipelined parallelism.

Async boundaries are where backpressure buffers actually live.

throttle imposes a deliberate maximum rate, generating backpressure upstream to honor it. This protects rate-limited external services even when the consumer could go faster.

A burst parameter allows short spikes above the steady rate.

You can detect backpressure by watching for upstream slowdowns or by measuring buffer occupancy. The log operator and Akka's stream attributes help trace where a pipeline stalls.

A persistently full buffer indicates the slowest stage that is gating throughput.

Think about how Akka Streams keeps a fast producer from flooding a slow consumer.

Backpressure is the demand-driven backbone of Akka Streams: consumers signal demand upstream so producers cannot overwhelm them, keeping memory bounded.

You saw internal buffers, the explicit buffer operator with overflow strategies, summarizing with conflate, filling demand with expand, async boundaries, and deliberate rate control via throttle. Next you will run a complete pipeline.