PricingLoginStart free
Advanced Spring Boot 4: Event-Driven Architecture (Kafka) · Lección

Confirmación manual de offsets

Implemente un control manual de las confirmaciones de offsets para controlar con precisión las garantías de procesamiento de mensajes y evitar la pérdida o duplicación de datos.

Lección 1 de 412 pasos

Confirmación manual de offsets es una lección gratuita de Advanced Spring Boot 4: Event-Driven Architecture (Kafka) en CoddyKit. Esta es la lección 1 de 4. Puedes leer la lección completa abajo gratuitamente — luego la practicas en el navegador con un editor de código integrado y un tutor de IA 24/7. Forma parte de la ruta de aprendizaje de Advanced Spring Boot 4: Event-Driven Architecture (Kafka), y tu progreso se sincroniza en la web y la app de CoddyKit. El curso de Advanced Spring Boot 4: Event-Driven Architecture (Kafka) incluye 4 lecciones en total.

Partes de esta lección aún no han sido traducidas y se muestran en inglés.

Why Manual Commits?

In Kafka, an offset marks the last message a consumer group has successfully processed from a topic partition. Committing an offset tells Kafka: "I've handled messages up to this point."

By default, Spring Kafka uses auto-commit, where offsets are committed periodically in the background. While convenient, this can sometimes lead to data loss or duplication if your application crashes mid-processing.

Manual offset committing gives you precise control, allowing you to decide exactly when an offset is marked as processed. This is crucial for ensuring message processing guarantees.

Auto-Commit: A Quick Look

With auto-commit, Kafka automatically commits offsets at a set interval (e.g., every 5 seconds). This means:

  • Messages are processed.
  • Offsets are committed later by Kafka.

If your application processes a message but crashes *before* Kafka's auto-commit interval passes, that message's offset might not be committed. When the application restarts, it will re-read and re-process that message, leading to potential duplicates (at-least-once processing).

Switching to Manual Mode

To take control of offset management, you need to disable auto-commit in your Spring Boot application's Kafka configuration. This is typically done by setting the AckMode.

The AckMode determines when a consumer acknowledges messages. For manual control, we'll use MANUAL_IMMEDIATE or MANUAL.

Here's how you might configure it in application.properties:

spring.kafka.consumer.enable-auto-commit=false
spring.kafka.listener.ack-mode=MANUAL_IMMEDIATE

The Acknowledgment Object

When ack-mode is set to a manual option, your @KafkaListener method can receive an additional parameter: the Acknowledgment object.

This object is your direct interface to signal to Kafka that you have successfully processed a message (or a batch of messages) and its offset can now be committed.

You'll call its acknowledge() method when you're ready.

Basic Manual Commit Example

Let's see a simple example where we manually commit the offset after processing each message. Notice the Acknowledgment acknowledgment parameter.

Run this code, then stop and restart. You'll see messages are not re-processed if committed.

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.kafka.support.Acknowledgment;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.CommandLineRunner;

@SpringBootApplication
public class ManualCommitApp implements CommandLineRunner {

    @Autowired
    private KafkaTemplate<String, String> kafkaTemplate;

    public static void main(String[] args) {
        SpringApplication.run(ManualCommitApp.class, args);
    }

    @Override
    public void run(String... args) throws Exception {
        System.out.println("Sending message...");
        kafkaTemplate.send("my-topic", "Hello CoddyKit!");
        System.out.println("Message sent.");
    }

    @KafkaListener(topics = "my-topic", groupId = "manual-group")
    public void listen(String message, Acknowledgment acknowledgment) {
        System.out.println("Received: " + message);
        // Simulate processing
        try {
            Thread.sleep(500);
        } catch (InterruptedException e) {
            Thread.currentThread().interrupt();
        }
        System.out.println("Processed: " + message + ", committing offset.");
        acknowledgment.acknowledge(); // Manual commit
    }
}

Configuring for the Example

For the previous example to work, you'd need a src/main/resources/application.properties file with Kafka broker details and the manual ack mode:

  • spring.kafka.bootstrap-servers=localhost:9092 (or your Kafka broker)
  • spring.kafka.consumer.enable-auto-commit=false
  • spring.kafka.listener.ack-mode=MANUAL_IMMEDIATE
  • spring.kafka.producer.key-serializer=org.apache.kafka.common.serialization.StringSerializer
  • spring.kafka.producer.value-serializer=org.apache.kafka.common.serialization.StringSerializer
  • spring.kafka.consumer.key-deserializer=org.apache.kafka.common.serialization.StringDeserializer
  • spring.kafka.consumer.value-deserializer=org.apache.kafka.common.serialization.StringDeserializer

Remember to have a local Kafka running!

When to Acknowledge?

The core principle for manual committing is: commit only after your business logic has successfully completed.

  • After each message: As shown in the previous example. Good for low throughput or critical messages.
  • After a batch: Process multiple messages, then commit once for the entire batch. This is more efficient for high throughput.
  • After external interactions: If you write data to a database, commit the offset *only after* the database transaction is successful.

Choosing the right strategy depends on your application's requirements for performance and data consistency.

Batch Processing & Manual Commit

When your listener consumes a batch of messages (e.g., List<String>), you should commit the offset only after *all* messages in that batch have been successfully processed. The Acknowledgment object still works for the entire batch.

This is often combined with AckMode.BATCH, though MANUAL_IMMEDIATE also works.

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.kafka.support.Acknowledgment;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.CommandLineRunner;
import java.util.List;

@SpringBootApplication
public class BatchManualCommitApp implements CommandLineRunner {

    @Autowired
    private KafkaTemplate<String, String> kafkaTemplate;

    public static void main(String[] args) {
        SpringApplication.run(BatchManualCommitApp.class, args);
    }

    @Override
    public void run(String... args) throws Exception {
        System.out.println("Sending 3 messages...");
        kafkaTemplate.send("my-batch-topic", "Batch Msg 1");
        kafkaTemplate.send("my-batch-topic", "Batch Msg 2");
        kafkaTemplate.send("my-batch-topic", "Batch Msg 3");
        System.out.println("Messages sent.");
    }

    // Ensure spring.kafka.listener.ack-mode=MANUAL_IMMEDIATE in properties
    @KafkaListener(topics = "my-batch-topic", groupId = "batch-manual-group")
    public void listenBatch(List<String> messages, Acknowledgment acknowledgment) {
        System.out.println("Received batch of " + messages.size() + " messages.");
        for (String msg : messages) {
            System.out.println("  Processing: " + msg);
            // Simulate processing each message in the batch
            try {
                Thread.sleep(100);
            } catch (InterruptedException e) {
                Thread.currentThread().interrupt();
            }
        }
        System.out.println("Finished processing batch. Committing offset.");
        acknowledgment.acknowledge(); // Commit once for the entire batch
    }
}

Handling Errors & Reprocessing

What happens if an error occurs during message processing *before* acknowledgment.acknowledge() is called?

Since the offset was not committed, Kafka considers the message (or batch) as not processed. Upon restart or rebalance, the consumer will re-fetch and re-process those messages. This is the foundation of at-least-once processing semantics.

While this guarantees no data loss, it requires your message processing logic to be idempotent, meaning processing the same message multiple times has the same effect as processing it once.

Trade-offs & Best Practices

Manual offset committing offers control but comes with considerations:

  • Overhead: Committing too frequently can add network and Kafka broker overhead.
  • Reprocessing Scope: Committing too infrequently means more messages might be reprocessed if a failure occurs.
  • Idempotency: Always design your consumers to be idempotent when using manual commits to handle potential duplicates gracefully.
  • Error Handling: Combine manual commits with robust exception handling (e.g., retries, Dead Letter Topics) to manage failures effectively.

Quick Check: Manual Commits

You are using manual offset committing in your Spring Kafka consumer. If an error occurs while processing a message, and the acknowledgment.acknowledge() method is NOT called for that message, what will happen?

Responde en la lección interactiva

Recap: Manual Offset Control

We've explored manual offset committing, a powerful technique for precise control over message processing in Spring Kafka.

  • It disables auto-commit, giving you control.
  • You use the Acknowledgment object to explicitly commit offsets.
  • Committing should happen only after successful business logic execution.
  • It enables at-least-once processing, but requires idempotent consumers.

This fine-grained control is vital for building robust and reliable event-driven applications.

Gratis para empezar

Aprende Advanced Spring Boot 4: Event-Driven Architecture (Kafka) con un tutor de IA — gratis

Escribe y ejecuta código real en tu navegador, obtén ayuda instantánea de un tutor de IA disponible 24/7 y continúa donde lo dejaste en la web o en la aplicación.

Cursos
12
Lecciones
48
Empezar a aprender gratisVer el curso completo

Preguntas frecuentes

¿La lección «Confirmación manual de offsets» es gratis?

Sí — el texto completo de «Confirmación manual de offsets» es gratis para leer aquí en la web. Para practicarla de forma interactiva (editor de código integrado y tutor de IA 24/7) y desbloquear el resto del curso de Advanced Spring Boot 4: Event-Driven Architecture (Kafka), actualiza a CoddyKit PRO. El curso de Advanced Spring Boot 4: Event-Driven Architecture (Kafka) incluye 4 lecciones en total.

¿Qué aprenderé en «Confirmación manual de offsets»?

Implemente un control manual de las confirmaciones de offsets para controlar con precisión las garantías de procesamiento de mensajes y evitar la pérdida o duplicación de datos. Practicas Advanced Spring Boot 4: Event-Driven Architecture (Kafka) con código real que ejecutas directamente en el navegador, y un tutor de IA 24/7 responde tus preguntas mientras trabajas en la lección.

¿Necesito experiencia previa para empezar Advanced Spring Boot 4: Event-Driven Architecture (Kafka)?

No se requiere experiencia previa. Advanced Spring Boot 4: Event-Driven Architecture (Kafka) en CoddyKit está estructurado para principiantes hasta estudiantes avanzados, así que puedes empezar aquí o desde el inicio y avanzar a tu ritmo. Esta es la lección 1 de 4.

¿Cuánto tiempo toma la lección «Confirmación manual de offsets»?

La mayoría de las lecciones de CoddyKit toman alrededor de 5–10 minutos. Cada una es compacta e interactiva, así que avanzas constantemente y retomas exactamente por donde dejaste en la web y la app.

¿Puedo escribir y ejecutar código en esta lección de Advanced Spring Boot 4: Event-Driven Architecture (Kafka)?

Sí. Cada lección de Advanced Spring Boot 4: Event-Driven Architecture (Kafka) incluye un editor de código integrado, así que escribes y ejecutas código real directamente en tu navegador y obtienes retroalimentación instantánea de IA — sin configuración local necesaria.

Todas las lecciones de este curso

  1. Confirmación manual de offsets
  2. Pausar y reanudar consumidores
  3. Concurrencia y gestión de hilos
  4. Listeners de reequilibrio y pertenencia estática
← Volver a Advanced Spring Boot 4: Event-Driven Architecture (Kafka)