Customizing Producer Configurations
Explore various producer configurations like acks, batch size, linger.ms, and retries to optimize performance and reliability.
Customizing Producer Configurations is a free Advanced Spring Boot 4: Event-Driven Architecture (Kafka) lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Advanced Spring Boot 4: Event-Driven Architecture (Kafka) learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Why Customize Producer Settings?
When sending messages to Kafka, default settings might not always be ideal. Customizing your producer's configuration is key to optimizing for specific needs.
You can fine-tune your producer for better performance, increased reliability, or optimized throughput, depending on your application's requirements.
Configuring in Spring Boot
In Spring Boot, Kafka producer properties are typically defined in your application.yml (or .properties) file.
All producer-related settings usually start with the prefix spring.kafka.producer. Spring Boot automatically picks these up to configure your KafkaTemplate.
spring:
kafka:
bootstrap-servers: localhost:9092
producer:
key-serializer: org.apache.kafka.common.serialization.StringSerializer
value-serializer: org.apache.kafka.common.serialization.StringSerializer
# Custom configurations go hereAcknowledgments (`acks`)
The acks (acknowledgments) property determines the level of durability for messages sent by the producer. It controls how many replicas must acknowledge the write before the producer considers the message sent successfully.
acks=0: Producer doesn't wait for any acknowledgment. Fastest, but lowest durability (messages might be lost).acks=1: Producer waits for the leader replica to acknowledge the write. Good balance of durability and speed.acks=all: Producer waits for all in-sync replicas to acknowledge. Highest durability, but slowest.
Setting `acks=all`
For critical messages where data loss is unacceptable, you'd typically set acks to all. This ensures that your message is safely replicated before the producer moves on.
Remember, higher durability often means slightly higher latency.
spring:
kafka:
producer:
acks: all # Ensures high durability
# Other producer configsBatching: `batch.size` & `linger.ms`
To improve throughput, Kafka producers don't send every message individually. Instead, they batch multiple messages together.
batch.size: The maximum amount of data (in bytes) that will be collected before sending a batch. Default is 16KB.linger.ms: The maximum time (in milliseconds) the producer will wait for more messages to accumulate in a batch. Default is 0ms (send immediately).
These two properties work together: a batch is sent when either batch.size is reached OR linger.ms expires.
Optimizing Batch Settings
Adjusting batch.size and linger.ms can significantly impact performance. Larger batches and longer linger times can increase throughput but also slightly increase latency for individual messages.
Here's how you might configure them for better batching:
spring:
kafka:
producer:
batch-size: 32768 # 32 KB (larger batch)
linger-ms: 50 # Wait up to 50ms (longer wait)
# Other producer configsRetries (`retries`)
What happens if a message fails to send due to a transient network issue or a temporary broker unavailability?
The retries property specifies how many times the producer should re-attempt sending a message that failed due to a potentially recoverable error. This greatly enhances the reliability of your message delivery.
Setting `retries`
Setting a reasonable number of retries helps ensure your messages eventually reach Kafka, even if there are temporary glitches.
It's often combined with delivery.timeout.ms, which defines the total time a producer will wait for a message to be delivered, including retries.
spring:
kafka:
producer:
retries: 5 # Try up to 5 times on failure
delivery-timeout-ms: 120000 # 2 minutes total timeout
# Other producer configsCustom Producer in Action
While Spring Boot handles the underlying Kafka client configuration based on your application.yml, let's see how these properties are set directly in a Java Kafka producer. This helps understand the core mechanism.
This example explicitly configures and uses a Kafka producer:
import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerRecord;
import java.util.Properties;
public class CustomProducerDemo {
public static void main(String[] args) {
Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9092");
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
// Custom configurations we discussed
props.put("acks", "all");
props.put("batch.size", 32768);
props.put("linger.ms", 50);
props.put("retries", 5);
try (KafkaProducer<String, String> producer = new KafkaProducer<>(props)) {
producer.send(new ProducerRecord<>("custom-topic", "key1", "Hello from CoddyKit!"));
producer.send(new ProducerRecord<>("custom-topic", "key2", "Another custom message!"));
producer.flush(); // Ensure all buffered records are sent
System.out.println("Messages sent with custom configurations.");
} catch (Exception e) {
e.printStackTrace();
}
}
}Check Your Knowledge
You've learned about several key producer configurations. Let's test your understanding!
Recap: Customizing Producers
Great job! You've explored how to customize your Kafka producers for various needs:
acks: Controls message durability.batch.size&linger.ms: Optimize for throughput by batching messages.retries: Enhances reliability by re-sending failed messages.
By understanding and adjusting these properties, you can tailor your Spring Boot Kafka applications to meet specific performance and reliability goals. Next, we'll dive into implementing Kafka consumers!
Frequently asked questions
Is the “Customizing Producer Configurations” lesson free?
Yes — the full text of “Customizing Producer Configurations” is free to read here on the web, and the Advanced Spring Boot 4: Event-Driven Architecture (Kafka) course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Advanced Spring Boot 4: Event-Driven Architecture (Kafka) course, upgrade to CoddyKit PRO.
What will I learn in “Customizing Producer Configurations”?
Explore various producer configurations like acks, batch size, linger.ms, and retries to optimize performance and reliability. You practise Advanced Spring Boot 4: Event-Driven Architecture (Kafka) with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Advanced Spring Boot 4: Event-Driven Architecture (Kafka)?
No prior experience is required. Advanced Spring Boot 4: Event-Driven Architecture (Kafka) on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Customizing Producer Configurations” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Advanced Spring Boot 4: Event-Driven Architecture (Kafka) lesson?
Yes. Every Advanced Spring Boot 4: Event-Driven Architecture (Kafka) lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Integrating Spring Kafka Starter
- Sending Messages with KafkaTemplate
- Customizing Producer Configurations
- Handling Producer Send Callbacks and Acknowledgments