Kafka Architecture Overview
Understand Kafka's distributed architecture, including brokers, Zookeeper, and the role of logs and segments in data storage.
Welcome to Kafka Architecture!
Ever wondered how massive companies handle huge streams of data? That's where Apache Kafka shines! It's a powerful, distributed streaming platform.
In this lesson, we'll peel back the layers to understand Kafka's core architecture. We'll explore its main components and how they work together.
The Brains: Kafka Brokers
At the heart of a Kafka cluster are brokers. Think of them as individual Kafka servers. A Kafka cluster is made up of one or more brokers.
- Store Data: Brokers receive and store messages (called events).
- Serve Clients: They handle requests from producers (apps sending data) and consumers (apps reading data).
- Distributed: For reliability and scalability, Kafka typically runs with multiple brokers.
All lessons in this course
- Kafka Architecture Overview
- Topics, Partitions, and Offsets
- Setting up Local Kafka with Docker
- Consumer Groups and Rebalancing