Kafka Core Concepts: Brokers, Topics

Let's meet Kafka's two fundamental building blocks: brokers and topics. These are how Kafka stores and organizes your data streams.

A Kafka broker is a single server running Kafka. It receives messages from producers, stores them on disk, and serves them to consumers — the engine of it all.

Real setups run multiple brokers together as a Kafka cluster. That gives you high availability (one broker fails, others cover) and scalability for more data.

A topic is a named category or feed where records are published. Different data — "user activity," "payments" — goes into different topics to stay organized.

Think of a topic like a named folder or DB table: logins go to user_logins, orders to product_orders. Apps quickly find the stream they need.

Every message (or record) is published to a specific topic, and apps subscribe to read it. It's publish-subscribe — a topic is the pipeline for one data stream.

Topics live on the brokers. A topic's data is spread across one or more brokers in the cluster, and that distribution is key to Kafka's scale and fault tolerance.

Together, brokers and topics give you scalability, durable on-disk storage, clean organization, and decoupling — producers and consumers only share the topic.

To make a stream of sensor data, you'd create a topic like sensor_readings, and Kafka readies its brokers to store and manage those messages.

Let's test your understanding of Kafka's core components!

You've got Kafka's two foundations: brokers (servers that store and serve data) and topics (logical streams hosted across them). Solid base.