Advanced Spring Boot 4: Event-Driven Architecture (Kafka) · レッスン

コンシューマーグループの管理

コンシューマーグループによって並列処理とフォールトトレランスが実現し、各グループでメッセージが一度だけ処理される仕組みを理解します。

レッスン 2/411 ステップ

「コンシューマーグループの管理」はCoddyKit上の無料Advanced Spring Boot 4: Event-Driven Architecture (Kafka)レッスンです。これはレッスン2/4です。下記で完全なレッスンを無料で読むことができます。その後、ブラウザ内の組み込みコードエディタと24時間対応のAIチューターでハンズオン演習できます。これはAdvanced Spring Boot 4: Event-Driven Architecture (Kafka)学習パスの一部であり、ウェブとCoddyKitアプリ全体で進捗が同期されます。 Advanced Spring Boot 4: Event-Driven Architecture (Kafka)コースには全4レッスンが含まれています。

このレッスンの一部はまだ翻訳されておらず、英語で表示されています。

What are Consumer Groups?

Welcome! Today, we'll explore Consumer Groups in Kafka. These are a core concept for scaling message consumption and ensuring fault tolerance.

Think of a consumer group as a team of consumers working together to process messages from one or more topics. Each message from a topic's partitions is delivered to only one consumer within that specific group.

Scaling & Fault Tolerance

Consumer groups solve two major challenges:

Scaling: By distributing partitions across multiple consumers, you can process messages much faster, increasing your application's throughput.
Fault Tolerance: If a consumer fails or leaves the group, Kafka automatically reassigns its partitions to other active consumers in the same group. This ensures continuous message processing.

Ultimately, a message is processed exactly once per consumer group, even with multiple consumers.

Partitions & Consumers

To understand groups, remember that Kafka topics are divided into partitions. These partitions are the unit of parallelism.

Within a consumer group, each partition is assigned to at most one consumer.
If you have more consumers than partitions in a group, some consumers will be idle.
If you have fewer consumers than partitions, some consumers will handle multiple partitions.

This assignment strategy ensures ordered processing within each partition while allowing parallel processing across partitions.

Dynamic Partition Rebalancing

Kafka is smart! When a consumer joins or leaves a group (e.g., an application starts, stops, or crashes), Kafka automatically triggers a rebalance.

During a rebalance, partitions are dynamically re-assigned among the active consumers in the group. This ensures all partitions continue to be consumed and adapts to changes in your application's scaling needs.

Rebalancing is an automatic process that guarantees high availability and adapts to fluctuating workloads.

`group.id` in Spring Kafka

In Spring Kafka, you define which consumer group your listener belongs to using the group.id property.

This unique string identifies your consumer group to Kafka. All consumers (whether in the same application instance or different instances) that share the exact same group.id are considered part of the same group.

You typically configure this in your application.properties or directly in the @KafkaListener annotation.

Spring Listener Example

Here's a basic Spring Boot Kafka listener configured with a specific group.id. This tells Kafka that this listener is part of the 'my-first-group' consumer group.

Try running this example and observe the output!

package com.coddykit;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.stereotype.Component;

@SpringBootApplication
public class ConsumerGroupApp {

    public static void main(String[] args) {
        SpringApplication.run(ConsumerGroupApp.class, args);
    }

    @Component
    public static class MyKafkaListener {

        @KafkaListener(topics = "my-topic", groupId = "my-first-group")
        public void listen(String message) {
            System.out.println("Received in Group 1: " + message);
        }
    }
}

Scaling with Multiple Consumers

To achieve parallel processing and higher throughput for a topic, you can run multiple instances of your application, all configured with the same group.id.

Kafka will then distribute the topic's partitions across these running instances. Each instance will process a subset of the partitions independently, effectively scaling out your message consumption.

This is the primary way to handle high-volume topics efficiently.

Multiple Listeners, Same Group

You can also define multiple @KafkaListener methods within the same Spring Boot application instance, all belonging to the same consumer group.

Spring Kafka manages these as separate consumers within that group. If the topic has enough partitions, these listeners can process messages from different partitions in parallel within a single application.

The id attribute helps differentiate listener beans, especially when multiple listeners are defined for the same topic and group.

package com.coddykit;

import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.stereotype.Component;

@SpringBootApplication
public class MultiListenerApp {

    public static void main(String[] args) {
        SpringApplication.run(MultiListenerApp.class, args);
    }

    @Component
    public static class MyMultiKafkaListeners {

        // Consumer 1 for "my-topic" in "my-app-group"
        @KafkaListener(topics = "my-topic", groupId = "my-app-group", id = "listener1")
        public void listen1(String message) {
            System.out.println("Listener 1 received: " + message);
        }

        // Consumer 2 for "my-topic" in "my-app-group"
        @KafkaListener(topics = "my-topic", groupId = "my-app-group", id = "listener2")
        public void listen2(String message) {
            System.out.println("Listener 2 received: " + message);
        }
    }
}

Independent Consumption

What if different applications need to process the same messages from a topic, but for different purposes?

This is where multiple consumer groups come in handy. You can have several distinct consumer groups, each with its own unique group.id, consuming from the same Kafka topic.

Each group will receive a full copy of all messages published to that topic. This allows for independent processing without affecting other groups.

Group Management Check

Consider a Kafka topic with 3 partitions. You start an application with one consumer group and two active consumers. How will messages be distributed?

対話的なレッスンで答える

Consumer Group Summary

You've learned about the power of Kafka Consumer Groups!

They are crucial for scaling message consumption and building fault-tolerant applications.
Messages from a topic's partitions are distributed among consumers in a group, processed exactly once per group.
Kafka handles partition assignment and rebalancing automatically as consumers join or leave.
Using different group.id values enables multiple applications to independently consume the same stream of messages.

Mastering consumer groups is key to building robust Kafka applications.

無料で開始

AI チューターと学ぶ Advanced Spring Boot 4: Event-Driven Architecture (Kafka) — 無料

ブラウザでリアルコードを書いて実行し、24/7 の AI チューターから瞬時にサポートを受け、ウェブまたはアプリで続きから学習できます。

コース: 12
レッスン: 48

無料で学習を始めるコース全体を表示

よくある質問

「コンシューマーグループの管理」レッスンは無料ですか？

はい。「コンシューマーグループの管理」の完全なテキストはこのウェブで無料で読めます。インタラクティブに演習し（組み込みコードエディタと24時間対応のAIチューター）、Advanced Spring Boot 4: Event-Driven Architecture (Kafka)コースの残りをアンロックするには、CoddyKit PROにアップグレードしてください。 Advanced Spring Boot 4: Event-Driven Architecture (Kafka)コースには全4レッスンが含まれています。

「コンシューマーグループの管理」で何を学びますか？

コンシューマーグループによって並列処理とフォールトトレランスが実現し、各グループでメッセージが一度だけ処理される仕組みを理解します。ブラウザで直接実行するハンズオンコードでAdvanced Spring Boot 4: Event-Driven Architecture (Kafka)を演習し、24時間対応のAIチューターがレッスンを進める中での質問に答えます。

Advanced Spring Boot 4: Event-Driven Architecture (Kafka)を始めるのに経験は必要ですか？

事前経験は必要ありません。CoddyKitのAdvanced Spring Boot 4: Event-Driven Architecture (Kafka)は初級者から上級者向けに構成されているため、ここから始めるか最初から始めて、自分のペースで進むことができます。これはレッスン2/4です。

「コンシューマーグループの管理」レッスンにはどのくらい時間がかかりますか？

ほとんどのCoddyKitレッスンは約5～10分かかります。各レッスンはコンパクトでインタラクティブなので、着実に進歩し、ウェブとアプリ全体で正確に前回の場所から再開できます。

このAdvanced Spring Boot 4: Event-Driven Architecture (Kafka)レッスンでコードを書いて実行できますか？

はい。すべてのAdvanced Spring Boot 4: Event-Driven Architecture (Kafka)レッスンに組み込みコードエディタが含まれているため、ブラウザでリアルコードを書いて実行し、即座のAIフィードバックを取得できます。ローカル設定は不要です。

このコースのすべてのレッスン

← Advanced Spring Boot 4: Event-Driven Architecture (Kafka)に戻る