Monitoring Consumer Group Lag

In a healthy pipeline, consumers keep up with producers. When they fall behind, lag builds up: unread or unacknowledged entries pile in the stream. Monitoring lag lets you scale consumers before backlogs become outages.

XINFO STREAM gives a high-level view: length, last generated ID, and the number of consumer groups attached.

XINFO GROUPS lists each group with its last-delivered-id, the count of pending entries, and (in recent Redis) a lag field showing how many entries the group has not yet read.

The lag value is the number of entries between the group's last-delivered-id and the stream's last entry. A steadily rising lag means producers outpace consumers.

XINFO CONSUMERS drills into a single group, showing each consumer's pending count and idle time. A consumer with high idle time and many pending entries is likely stuck or dead.

XPENDING with just the key and group gives a summary: total pending, the lowest and highest pending IDs, and a per-consumer breakdown.

The extended form lists individual pending entries with their idle time and delivery count, helping you find messages that keep failing.

A high delivery count on a pending entry signals a poison message: one that repeatedly fails and gets re-delivered. Route it to a dead-letter stream after a threshold.

Turn metrics into alerts: warn when group lag exceeds a budget, or when any consumer's idle time crosses a limit while holding pending entries. Sample these via XINFO from a monitoring job.

When lag climbs, add consumers to the group to parallelize processing, optimize the per-message work, or increase the trim cap so the buffer can absorb spikes. Each consumer in a group gets a disjoint slice of new entries.

A simple monitor periodically calls XINFO GROUPS, extracts lag and pending, and emits the values to your metrics system for dashboards and alerts.

Test your understanding of lag monitoring.

You learned to monitor consumer group health using XINFO STREAM/GROUPS/CONSUMERS for lag and idle time, and XPENDING for pending detail and delivery counts. Use these signals to detect poison messages, set alert thresholds, and scale consumers before backlogs grow.