消费者组管理
了解消费者组如何实现并行处理和容错，确保每条消息在每个组内只被处理一次。
消费者组管理 是 CoddyKit 上的免费 Advanced Spring Boot 4: Event-Driven Architecture (Kafka) 课时。 这是第 2 节课，共 4 节。 你可以在下方免费阅读本课时的完整内容 — 然后在浏览器中使用内置代码编辑器和全天候 AI 导师进行实践。 这是 Advanced Spring Boot 4: Event-Driven Architecture (Kafka) 学习路径的一部分，你的进度在网页和 CoddyKit 应用中同步。 Advanced Spring Boot 4: Event-Driven Architecture (Kafka) 课程共包含 4 节课。
本课时的部分内容尚未翻译，以英文显示。
What are Consumer Groups?
Welcome! Today, we'll explore Consumer Groups in Kafka. These are a core concept for scaling message consumption and ensuring fault tolerance.
Think of a consumer group as a team of consumers working together to process messages from one or more topics. Each message from a topic's partitions is delivered to only one consumer within that specific group.
Scaling & Fault Tolerance
Consumer groups solve two major challenges:
- Scaling: By distributing partitions across multiple consumers, you can process messages much faster, increasing your application's throughput.
- Fault Tolerance: If a consumer fails or leaves the group, Kafka automatically reassigns its partitions to other active consumers in the same group. This ensures continuous message processing.
Ultimately, a message is processed exactly once per consumer group, even with multiple consumers.
Partitions & Consumers
To understand groups, remember that Kafka topics are divided into partitions. These partitions are the unit of parallelism.
- Within a consumer group, each partition is assigned to at most one consumer.
- If you have more consumers than partitions in a group, some consumers will be idle.
- If you have fewer consumers than partitions, some consumers will handle multiple partitions.
This assignment strategy ensures ordered processing within each partition while allowing parallel processing across partitions.
Dynamic Partition Rebalancing
Kafka is smart! When a consumer joins or leaves a group (e.g., an application starts, stops, or crashes), Kafka automatically triggers a rebalance.
During a rebalance, partitions are dynamically re-assigned among the active consumers in the group. This ensures all partitions continue to be consumed and adapts to changes in your application's scaling needs.
Rebalancing is an automatic process that guarantees high availability and adapts to fluctuating workloads.
`group.id` in Spring Kafka
In Spring Kafka, you define which consumer group your listener belongs to using the group.id property.
This unique string identifies your consumer group to Kafka. All consumers (whether in the same application instance or different instances) that share the exact same group.id are considered part of the same group.
You typically configure this in your application.properties or directly in the @KafkaListener annotation.
Spring Listener Example
Here's a basic Spring Boot Kafka listener configured with a specific group.id. This tells Kafka that this listener is part of the 'my-first-group' consumer group.
Try running this example and observe the output!
package com.coddykit;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.stereotype.Component;
@SpringBootApplication
public class ConsumerGroupApp {
public static void main(String[] args) {
SpringApplication.run(ConsumerGroupApp.class, args);
}
@Component
public static class MyKafkaListener {
@KafkaListener(topics = "my-topic", groupId = "my-first-group")
public void listen(String message) {
System.out.println("Received in Group 1: " + message);
}
}
}Scaling with Multiple Consumers
To achieve parallel processing and higher throughput for a topic, you can run multiple instances of your application, all configured with the same group.id.
Kafka will then distribute the topic's partitions across these running instances. Each instance will process a subset of the partitions independently, effectively scaling out your message consumption.
This is the primary way to handle high-volume topics efficiently.
Multiple Listeners, Same Group
You can also define multiple @KafkaListener methods within the same Spring Boot application instance, all belonging to the same consumer group.
Spring Kafka manages these as separate consumers within that group. If the topic has enough partitions, these listeners can process messages from different partitions in parallel within a single application.
The id attribute helps differentiate listener beans, especially when multiple listeners are defined for the same topic and group.
package com.coddykit;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.stereotype.Component;
@SpringBootApplication
public class MultiListenerApp {
public static void main(String[] args) {
SpringApplication.run(MultiListenerApp.class, args);
}
@Component
public static class MyMultiKafkaListeners {
// Consumer 1 for "my-topic" in "my-app-group"
@KafkaListener(topics = "my-topic", groupId = "my-app-group", id = "listener1")
public void listen1(String message) {
System.out.println("Listener 1 received: " + message);
}
// Consumer 2 for "my-topic" in "my-app-group"
@KafkaListener(topics = "my-topic", groupId = "my-app-group", id = "listener2")
public void listen2(String message) {
System.out.println("Listener 2 received: " + message);
}
}
}Independent Consumption
What if different applications need to process the same messages from a topic, but for different purposes?
This is where multiple consumer groups come in handy. You can have several distinct consumer groups, each with its own unique group.id, consuming from the same Kafka topic.
Each group will receive a full copy of all messages published to that topic. This allows for independent processing without affecting other groups.
Group Management Check
Consider a Kafka topic with 3 partitions. You start an application with one consumer group and two active consumers. How will messages be distributed?
Consumer Group Summary
You've learned about the power of Kafka Consumer Groups!
- They are crucial for scaling message consumption and building fault-tolerant applications.
- Messages from a topic's partitions are distributed among consumers in a group, processed exactly once per group.
- Kafka handles partition assignment and rebalancing automatically as consumers join or leave.
- Using different
group.idvalues enables multiple applications to independently consume the same stream of messages.
Mastering consumer groups is key to building robust Kafka applications.
用 AI 导师学习 Advanced Spring Boot 4: Event-Driven Architecture (Kafka) — 免费
在浏览器中编写并运行真实代码，获得全天候 AI 导师的即时帮助，并在网页或应用中继续学习。
- 课程
- 12
- 课程
- 48
常见问题解答
「消费者组管理」课时是免费的吗？
是的 — 「消费者组管理」的完整文本可在网页上免费阅读。要进行交互式练习（内置代码编辑器和全天候 AI 导师）并解锁 Advanced Spring Boot 4: Event-Driven Architecture (Kafka) 课程的其余内容，请升级到 CoddyKit PRO。 Advanced Spring Boot 4: Event-Driven Architecture (Kafka) 课程共包含 4 节课。
「消费者组管理」这节课中我会学到什么？
了解消费者组如何实现并行处理和容错，确保每条消息在每个组内只被处理一次。 你通过在浏览器中直接运行的动手代码来练习 Advanced Spring Boot 4: Event-Driven Architecture (Kafka)，全天候 AI 导师会在你学习这节课的过程中回答你的问题。
学习 Advanced Spring Boot 4: Event-Driven Architecture (Kafka) 需要有经验吗？
无需任何先前经验。CoddyKit 上的 Advanced Spring Boot 4: Event-Driven Architecture (Kafka) 课程适合初学者到高级学习者，你可以从这里开始或从头开始，按照自己的节奏学习。 这是第 2 节课，共 4 节。
「消费者组管理」课时需要多长时间？
大多数 CoddyKit 课程大约需要 5–10 分钟。每节课都很精短且互动，所以你能稳步进步，并在网页和应用中从离开的地方继续。
我能在这节 Advanced Spring Boot 4: Event-Driven Architecture (Kafka) 课中编写并运行代码吗？
能。每节 Advanced Spring Boot 4: Event-Driven Architecture (Kafka) 课都包含内置代码编辑器，你可以在浏览器中直接编写并运行真实代码，并获得即时 AI 反馈 — 无需本地设置。