مراقبة تأخر المستهلكين وإعداد التنبيهات

Consumer lag is the difference between the latest offset produced to a partition (the log-end offset) and the offset a consumer group has committed.

High lag means consumers are falling behind producers.

Lag is the single most actionable health signal for streaming systems. Growing lag predicts delayed processing, stale data, and eventually full disks if retention is exceeded.

Lag is measured per partition and summed per group. One slow partition can hide behind a healthy total, so always inspect per-partition lag too.

The bundled tool reports lag for a group quickly during incidents.

Spring Kafka publishes consumer metrics through Micrometer. The records-lag-max metric is exported to your registry for scraping.

Prometheus scrapes the /actuator/prometheus endpoint, capturing the lag gauge over time so you can graph trends.

For accurate group-level lag and an estimated time lag (how many seconds behind), dedicated exporters like Kafka Lag Exporter compute lag from committed offsets directly.

Alert on sustained, growing lag, not a single spike. A burst is normal after a deploy; a steady climb is a real problem.

Offset lag (10k records) means little without context. Time lag (5 minutes behind) is more meaningful for SLAs and easier to reason about.

When lag alerts fire, options include scaling out consumers (up to the partition count), increasing max.poll.records, or optimizing slow processing logic.

Lag monitoring turns invisible backpressure into a clear signal. Export the metric, graph per-partition trends, alert on sustained growth, and scale or optimize in response.

Test your understanding of consumer lag.

You learned to monitor consumer lag.

• Lag = log-end offset minus committed offset.
• Inspect per-partition lag, not just totals.
• Export it via Micrometer/Prometheus or a lag exporter.
• Alert on sustained growth and prefer time lag for SLAs.