Advanced Spring Boot 4: Event-Driven Architecture (Kafka) · درس

تنفيذ مواضيع الرسائل الميتة (DLT)

اضبطوا مواضيع الرسائل الميتة لالتقاط الرسائل التي تفشل معالجتها مرارًا وتخزينها، مما يتيح تحليلها وإعادة معالجتها لاحقًا

تنفيذ مواضيع الرسائل الميتة (DLT) درس مجاني في Advanced Spring Boot 4: Event-Driven Architecture (Kafka) على CoddyKit. هذا هو الدرس 3 من أصل 4. يمكنك قراءة الدرس كاملاً أدناه مجاناً — ثم تمرن عليه مباشرة في المتصفح باستخدام محرر أكواد مدمج ومدرس ذكاء اصطناعي متاح 24/7. هذا الدرس جزء من مسار التعلم في Advanced Spring Boot 4: Event-Driven Architecture (Kafka)، وتقدمك يتزامن عبر الويب وتطبيق CoddyKit. تتضمن دورة Advanced Spring Boot 4: Event-Driven Architecture (Kafka) 4 دروس في المجموع.

بعض أجزاء هذا الدرس لم تُترجم بعد وتظهر باللغة الإنجليزية.

Why Dead Letter Topics?

In event-driven systems, consumers sometimes fail to process messages due to transient errors (e.g., database unavailable) or permanent issues (e.g., malformed data).

What happens to these failed messages?
Do we retry them indefinitely, blocking the queue?
Or do we discard them, potentially losing critical data?

This is where Dead Letter Topics (DLTs) come in!

What is a Dead Letter Topic?

A Dead Letter Topic (DLT) is a dedicated Kafka topic where messages that repeatedly fail processing are sent.

It acts as a 'quarantine' for problematic messages.
Instead of blocking the main consumer or losing data, messages are moved to the DLT.
This allows the main consumer to continue processing new messages.

Messages in a DLT can then be inspected, manually corrected, or reprocessed later.

Spring Kafka's DLT Support

Spring for Apache Kafka provides excellent support for DLTs through its error handling mechanisms.

The key component is the DefaultErrorHandler, which can be configured to publish failed messages to a DLT after a certain number of retries.

It uses a DeadLetterPublishingRecoverer internally to perform the actual publishing.

Basic DLT Configuration

To enable DLT for a @KafkaListener, you can configure the DefaultErrorHandler with a DeadLetterPublishingRecoverer. This example sets up a simple DLT after 3 delivery attempts.

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.kafka.listener.DeadLetterPublishingRecoverer;
import org.springframework.kafka.listener.DefaultErrorHandler;
import org.springframework.util.backoff.FixedBackOff;

@SpringBootApplication
public class DltApplication {

  public static void main(String[] args) {
    SpringApplication.run(DltApplication.class, args);
  }

  @Bean
  public DefaultErrorHandler errorHandler(KafkaTemplate<?, ?> kafkaTemplate) {
    // Publish to DLT after 3 delivery attempts
    // FixedBackOff(interval, maxAttempts) -> interval is ignored for DLT after retries
    return new DefaultErrorHandler(new DeadLetterPublishingRecoverer(kafkaTemplate),
                                   new FixedBackOff(0L, 2L)); // 0L interval, 2 retries = 3 attempts total
  }

  @KafkaListener(topics = "my-main-topic", groupId = "my-group", errorHandler = "errorHandler")
  public void listen(String message) {
    System.out.println("Received: " + message);
    if (message.contains("fail")) {
      throw new RuntimeException("Simulating processing failure!");
    }
  }

  // To send messages for testing (not part of DLT config itself)
  // @Autowired
  // private KafkaTemplate<String, String> template;
  // @EventListener(ApplicationReadyEvent.class)
  // public void sendMessage() {
  //   template.send("my-main-topic", "Hello");
  //   template.send("my-main-topic", "This will fail");
  // }
}

Understanding DLT Topic Names

By default, Spring Kafka names the DLT topic by appending .DLT to the original topic name. For example, if your main topic is my-main-topic, the DLT will be my-main-topic.DLT.

You can customize this behavior.
The DeadLetterPublishingRecoverer constructor can take a BiFunction to determine the DLT topic and partition.
This allows for more flexible naming conventions or routing failed messages to different DLTs based on criteria.

Message Headers in DLT

When a message is sent to a DLT, Spring Kafka adds several useful headers to it. These headers provide context about why the message ended up in the DLT:

dlt_exception-fqcn: Fully qualified class name of the exception.
dlt_exception-message: Message from the exception.
dlt_exception-stacktrace: Full stack trace.
dlt_original-topic: The topic the message came from.
dlt_original-partition: The original partition.
dlt_original-offset: The original offset.

These headers are invaluable for debugging and reprocessing.

Customizing DLT Publishing

You can provide a custom DeadLetterPublishingRecoverer to gain fine-grained control over how messages are published to the DLT. This allows you to modify headers, filter messages, or even prevent certain messages from going to the DLT.

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.kafka.listener.DeadLetterPublishingRecoverer;
import org.springframework.kafka.listener.DefaultErrorHandler;
import org.springframework.kafka.support.KafkaHeaders;
import org.springframework.messaging.Message;
import org.springframework.messaging.support.MessageBuilder;
import org.springframework.util.backoff.FixedBackOff;

@SpringBootApplication
public class CustomDltApplication {

  public static void main(String[] args) {
    SpringApplication.run(CustomDltApplication.class, args);
  }

  @Bean
  public DefaultErrorHandler customErrorHandler(KafkaTemplate<Object, Object> kafkaTemplate) {
    DeadLetterPublishingRecoverer customRecoverer = new DeadLetterPublishingRecoverer(kafkaTemplate,
      (record, exception) -> { // Custom DLT topic/partition resolver
        System.out.println("Sending to DLT: " + record.topic() + ".custom.dlt");
        return new DeadLetterPublishingRecoverer.HeaderNames(record.topic() + ".custom.dlt", null);
      });
    return new DefaultErrorHandler(customRecoverer, new FixedBackOff(0L, 1L)); // 1 retry = 2 attempts total
  }

  @KafkaListener(topics = "another-topic", groupId = "my-custom-group", errorHandler = "customErrorHandler")
  public void listenWithCustomDlt(String message) {
    System.out.println("Received (custom DLT): " + message);
    if (message.contains("fail")) {
      throw new RuntimeException("Simulating custom DLT failure!");
    }
  }
}

Consuming DLT Messages

Once messages are in a DLT, you'll need another consumer to process them. This DLT consumer can be designed to:

Log the error and notify administrators.
Store the message in a database for manual review.
Attempt to reprocess the message after a delay or transformation.

It's just another @KafkaListener, but configured to listen to the DLT topic.

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.context.annotation.Bean;
import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.kafka.listener.DeadLetterPublishingRecoverer;
import org.springframework.kafka.listener.DefaultErrorHandler;
import org.springframework.messaging.handler.annotation.Header;
import org.springframework.kafka.support.KafkaHeaders;
import org.springframework.util.backoff.FixedBackOff;

@SpringBootApplication
public class DltConsumerApplication {

  public static void main(String[] args) {
    SpringApplication.run(DltConsumerApplication.class, args);
  }

  @Bean
  public DefaultErrorHandler errorHandler(KafkaTemplate<?, ?> kafkaTemplate) {
    return new DefaultErrorHandler(new DeadLetterPublishingRecoverer(kafkaTemplate),
                                   new FixedBackOff(0L, 2L));
  }

  @KafkaListener(topics = "my-main-topic", groupId = "my-group", errorHandler = "errorHandler")
  public void listenMain(String message) {
    System.out.println("Main Listener Received: " + message);
    if (message.contains("fail")) {
      throw new RuntimeException("Main processing failure!");
    }
  }

  @KafkaListener(topics = "my-main-topic.DLT", groupId = "dlt-group")
  public void listenDlt(String message,
                        @Header(KafkaHeaders.RECEIVED_TOPIC) String receivedTopic,
                        @Header(KafkaHeaders.ORIGINAL_OFFSET) Long originalOffset,
                        @Header(KafkaHeaders.EXCEPTION_MESSAGE) String exceptionMessage) {
    System.out.println("DLT Listener Received: " + message);
    System.out.println("  From Topic: " + receivedTopic);
    System.out.println("  Original Offset: " + originalOffset);
    System.out.println("  Exception: " + exceptionMessage);
    // Here you would implement logic to store, alert, or reprocess
  }
}

DLT Best Practices

To effectively use DLTs, consider these best practices:

Monitor DLTs: Set up alerts for messages appearing in DLTs, as they indicate persistent issues.
Process DLTs: Don't let DLTs grow indefinitely. Have a plan to consume and handle these messages.
Idempotency: Ensure your DLT reprocessing logic is idempotent to avoid duplicate processing issues.
Separate Concerns: Keep DLT consumers separate from your main application logic for clearer responsibilities.
Schema Evolution: Be mindful of schema changes when reprocessing old DLT messages.

DLT Quick Check

Which of the following is the primary benefit of using a Dead Letter Topic (DLT) in a Kafka consumer application?

Recap: DLT for Robustness

You've learned how Dead Letter Topics are a crucial component for building robust and resilient Kafka consumer applications.

DLTs quarantine messages that fail repeated processing.
Spring Kafka's DefaultErrorHandler and DeadLetterPublishingRecoverer simplify DLT integration.
Messages sent to DLTs include useful headers for debugging.
DLTs require a separate consumer to handle the failed messages.

By implementing DLTs, you ensure your consumers can gracefully handle errors, prevent data loss, and maintain steady message flow.

الأسئلة الشائعة

هل درس «تنفيذ مواضيع الرسائل الميتة (DLT)» مجاني؟

نعم — نص درس «تنفيذ مواضيع الرسائل الميتة (DLT)» كامل متاح مجاناً هنا على الويب. لتمرينه بشكل تفاعلي (محرر أكواد مدمج ومدرس ذكاء اصطناعي متاح 24/7) وفتح باقي دورة Advanced Spring Boot 4: Event-Driven Architecture (Kafka)، انتقل إلى CoddyKit PRO. تتضمن دورة Advanced Spring Boot 4: Event-Driven Architecture (Kafka) 4 دروس في المجموع.

ماذا ستتعلم في «تنفيذ مواضيع الرسائل الميتة (DLT)»؟

اضبطوا مواضيع الرسائل الميتة لالتقاط الرسائل التي تفشل معالجتها مرارًا وتخزينها، مما يتيح تحليلها وإعادة معالجتها لاحقًا تتمرن على Advanced Spring Boot 4: Event-Driven Architecture (Kafka) مع أكواد عملية تشغلها مباشرة في المتصفح، ومدرس ذكاء اصطناعي متاح 24/7 يجيب على أسئلتك أثناء عملك.

هل أحتاج إلى خبرة سابقة لأبدأ Advanced Spring Boot 4: Event-Driven Architecture (Kafka)؟

لا تُشترط خبرة سابقة. Advanced Spring Boot 4: Event-Driven Architecture (Kafka) على CoddyKit منظم للمبتدئين حتى المتقدمين، لذا يمكنك البدء من هنا أو من البداية والتقدم بسرعتك الخاصة. هذا هو الدرس 3 من أصل 4.

كم من الوقت يستغرق درس «تنفيذ مواضيع الرسائل الميتة (DLT)»؟

معظم دروس CoddyKit تستغرق حوالي 5–10 دقائق. كل منها موجز وتفاعلي، لذا تحرز تقدماً مستمراً وتستأنف من حيث توقفت عبر الويب والتطبيق.

هل يمكنني كتابة وتشغيل أكواد في درس Advanced Spring Boot 4: Event-Driven Architecture (Kafka) هذا؟

نعم. كل درس في Advanced Spring Boot 4: Event-Driven Architecture (Kafka) يتضمن محرر أكواد مدمج، لذا تكتب وتشغل أكواداً حقيقية مباشرة في متصفحك وتحصل على تعليقات فورية من الذكاء الاصطناعي — بدون إعداد محلي.

جميع الدروس في هذه الدورة

معالجة استثناءات المستهلكين
آليات إعادة المحاولة باستخدام Spring Retry
تنفيذ مواضيع الرسائل الميتة (DLT)
إعادات المحاولة غير الحاجبة باستخدام موضوعات إعادة المحاولة

← العودة إلى Advanced Spring Boot 4: Event-Driven Architecture (Kafka)