소비자 지연 추적 및 경고

Consumer lag is the difference between the latest offset produced to a partition (the log-end offset) and the offset a consumer group has committed.

Lag tells you how far behind your consumers are. Rising lag means consumers can't keep up with producers.

High lag has real consequences:

• Stale data downstream (dashboards, alerts, ML features).
• Risk of hitting retention and losing unconsumed messages.
• A signal of undersized consumers or a stuck partition.

The fastest way to see lag is kafka-consumer-groups.sh with --describe.

The describe output shows per-partition columns:

• CURRENT-OFFSET — last committed offset.
• LOG-END-OFFSET — newest offset in the partition.
• LAG — the difference between them.

Sum LAG across partitions for total group lag.

Each consumer exposes lag through JMX under the consumer-fetch-manager-metrics group.

• records-lag-max — max lag across assigned partitions.
• records-lag — per-partition lag.

These are client-side and update in near real time.

For production monitoring, dedicated tools poll lag for all groups:

• Burrow — evaluates lag trend and reports group status (OK/WARN/ERR).
• kafka_exporter — exposes lag as Prometheus metrics.

With kafka_exporter, lag is available as a labeled time series you can graph and alert on in Grafana.

Alert on sustained high lag, not momentary spikes. A common rule fires when total lag exceeds a threshold for several minutes.

Absolute lag alone can mislead — 10,000 messages may be seconds of data for a fast topic.

Track the rate of change: if lag keeps growing, consumers are losing the race even if current numbers look fine.

When lag climbs, your options include:

• Add consumers (up to the partition count).
• Increase partitions to allow more parallelism.
• Tune max.poll.records and processing efficiency.
• Check for a slow or stuck partition / poison message.

Make lag a first-class signal:

• Dashboard total and per-partition lag for every critical group.
• Alert on lag trend, not just thresholds.
• Keep lag well below the retention window so you never lose data.

You learned to track and alert on consumer lag.

• Lag = log-end offset minus committed offset.
• Inspect it via kafka-consumer-groups.sh, JMX, Burrow, or kafka_exporter.
• Alert on sustained lag and lag growth rate.
• Reduce lag by scaling consumers/partitions and tuning processing.