Batching, Compression & Linger Tuning
Learn how producer batching, linger.ms, and compression interact to trade latency for throughput and reduce network and disk usage.
The Throughput Lever
Sending one record per network call is slow. Kafka producers batch records per partition before sending, amortizing overhead and dramatically raising throughput.
batch.size
batch.size sets the maximum size (in bytes) of a single batch per partition.
- Larger batches = better throughput, more memory.
- A batch is sent when it fills or when linger expires.
// 64 KB batches
props.put(ProducerConfig.BATCH_SIZE_CONFIG, 65536);All lessons in this course
- Producer & Consumer Performance
- Broker Configuration & Tuning
- Disk I/O & Network Optimization
- Batching, Compression & Linger Tuning