Apache Kafka & Stream Processing Fundamentals · 课时

构建简单的 Kafka Streams 应用

创建您的第一个 Kafka Streams 应用，从 Kafka 主题实时处理数据

构建简单的 Kafka Streams 应用 是 CoddyKit 上的免费 Apache Kafka & Stream Processing Fundamentals 课时。这是第 1 节课，共 4 节。你可以在下方免费阅读本课时的完整内容 — 然后在浏览器中使用内置代码编辑器和全天候 AI 导师进行实践。这是 Apache Kafka & Stream Processing Fundamentals 学习路径的一部分，你的进度在网页和 CoddyKit 应用中同步。 Apache Kafka & Stream Processing Fundamentals 课程共包含 4 节课。

本课时的部分内容尚未翻译，以英文显示。

Welcome to Kafka Streams!

Ready to build real-time data processing apps? Kafka Streams is a client library for building applications and microservices that process data stored in Kafka.

It lets you write standard Java/Scala applications that leverage Kafka's power for stream processing. Think of it as a toolkit to transform, filter, and analyze data as it flows through Kafka.

The Power of Kafka Streams

Kafka Streams offers several key advantages for your real-time applications:

Lightweight: It's just a library, no separate cluster needed.
Fault-Tolerant: Automatically handles failures and data recovery.
Scalable: Easily scales by adding more instances of your application.
Exactly-Once Processing: Guarantees data is processed once, even with failures.

It's great for real-time analytics, data transformations, and event-driven microservices.

Setting Up Your Project

To start, you'll need to add the Kafka Streams library to your project. If you're using Maven, add this dependency to your pom.xml:

<dependency>
  <groupId>org.apache.kafka</groupId>
  <artifactId>kafka-streams</artifactId>
  <version>3.5.1</version>
</dependency>

Core Component: StreamsBuilder

The StreamsBuilder is your main entry point for defining the stream processing topology. Think of it as the architect for your data flow.

You use it to create source streams, apply transformations, and define where the processed data should go. Here's how you'd create one:

import org.apache.kafka.streams.StreamsBuilder;

public class Main {
  public static void main(String[] args) {
    StreamsBuilder builder = new StreamsBuilder();
    // Your stream processing logic will go here
    System.out.println("StreamsBuilder created!");
  }
}

KStream: Records in Motion

A KStream represents an unbounded, continuously updating stream of key-value records. Each record is processed independently as it arrives.

You can create a KStream from a Kafka topic using the stream() method of your StreamsBuilder. This tells your app to start consuming messages from that topic.

import org.apache.kafka.streams.StreamsBuilder;
import org.apache.kafka.streams.kstream.KStream;

public class Main {
  public static void main(String[] args) {
    StreamsBuilder builder = new StreamsBuilder();
    KStream<String, String> sourceStream =
        builder.stream("input-topic");
    System.out.println("KStream created from input-topic!");
  }
}

Simple Transformation: mapValues

One common operation is to transform the value of each record in a KStream. The mapValues() method is perfect for this.

It applies a function to each record's value, keeping the key unchanged. Let's write code to make all text values uppercase!

import org.apache.kafka.streams.StreamsBuilder;
import org.apache.kafka.streams.kstream.KStream;

public class Main {
  public static void main(String[] args) {
    StreamsBuilder builder = new StreamsBuilder();
    KStream<String, String> sourceStream =
        builder.stream("input-topic");

    KStream<String, String> transformedStream =
        sourceStream.mapValues(value -> value.toUpperCase());

    System.out.println("Stream values will be uppercased!");
  }
}

Essential Stream Configuration

Before running your app, you need to configure it. This is done using a Properties object and StreamsConfig. Key settings include:

APPLICATION_ID_CONFIG: Unique ID for your app (like a consumer group).
BOOTSTRAP_SERVERS_CONFIG: Your Kafka broker addresses.
DEFAULT_KEY_SERDE_CLASS_CONFIG: How to serialize/deserialize keys.
DEFAULT_VALUE_SERDE_CLASS_CONFIG: How to serialize/deserialize values.

Serdes (Serializer/Deserializer) are crucial for converting data to/from bytes.

Your First Kafka Streams App

Let's put everything together! This app will read messages from an 'input-topic', convert their values to uppercase, and then write the results to an 'output-topic'.

Remember to create these topics in your Kafka cluster before running this code!

import org.apache.kafka.common.serialization.Serdes;
import org.apache.kafka.streams.KafkaStreams;
import org.apache.kafka.streams.StreamsBuilder;
import org.apache.kafka.streams.StreamsConfig;
import org.apache.kafka.streams.kstream.KStream;

import java.util.Properties;

public class UppercaseStreamApp {

  public static void main(String[] args) {
    Properties props = new Properties();
    props.put(StreamsConfig.APPLICATION_ID_CONFIG, "uppercase-app");
    props.put(StreamsConfig.BOOTSTRAP_SERVERS_CONFIG, "localhost:9092");
    props.put(StreamsConfig.DEFAULT_KEY_SERDE_CLASS_CONFIG, Serdes.String().getClass());
    props.put(StreamsConfig.DEFAULT_VALUE_SERDE_CLASS_CONFIG, Serdes.String().getClass());

    StreamsBuilder builder = new StreamsBuilder();
    KStream<String, String> sourceStream = builder.stream("input-topic");

    KStream<String, String> transformedStream =
        sourceStream.mapValues(value -> value.toUpperCase());

    transformedStream.to("output-topic");

    KafkaStreams streams = new KafkaStreams(builder.build(), props);

    // Clean up local state on shutdown (for development)
    streams.cleanUp(); 
    
    streams.start();

    // Add shutdown hook to close Kafka Streams cleanly
    Runtime.getRuntime().addShutdownHook(new Thread(streams::close));
    System.out.println("UppercaseStreamApp started!");
  }
}

Managing Your Stream App

After defining your topology and configuration, you create a KafkaStreams instance and call start() to begin processing.

It's crucial to add a shutdown hook (Runtime.getRuntime().addShutdownHook) to ensure your application closes gracefully, flushing any buffered data and releasing resources.

The streams.cleanUp() call is useful during development to clear any local state store data, but should generally be avoided in production.

Test Your Knowledge

Which of the following is NOT a core component or essential configuration for a basic Kafka Streams application?

Recap: Your First Stream App

Great job! You've successfully learned the fundamentals of building a simple Kafka Streams application.

You now understand how to:

Add the necessary Kafka Streams dependency.
Use StreamsBuilder to define your processing topology.
Create a KStream from an input topic.
Apply simple transformations like mapValues().
Configure your application with StreamsConfig.
Start and gracefully stop your Kafka Streams application.

Next, we'll dive deeper into KStream and KTable concepts!

常见问题解答

「构建简单的 Kafka Streams 应用」课时是免费的吗？

是的 — 「构建简单的 Kafka Streams 应用」的完整文本可在网页上免费阅读。要进行交互式练习（内置代码编辑器和全天候 AI 导师）并解锁 Apache Kafka & Stream Processing Fundamentals 课程的其余内容，请升级到 CoddyKit PRO。 Apache Kafka & Stream Processing Fundamentals 课程共包含 4 节课。

「构建简单的 Kafka Streams 应用」这节课中我会学到什么？

创建您的第一个 Kafka Streams 应用，从 Kafka 主题实时处理数据你通过在浏览器中直接运行的动手代码来练习 Apache Kafka & Stream Processing Fundamentals，全天候 AI 导师会在你学习这节课的过程中回答你的问题。

学习 Apache Kafka & Stream Processing Fundamentals 需要有经验吗？

无需任何先前经验。CoddyKit 上的 Apache Kafka & Stream Processing Fundamentals 课程适合初学者到高级学习者，你可以从这里开始或从头开始，按照自己的节奏学习。这是第 1 节课，共 4 节。

「构建简单的 Kafka Streams 应用」课时需要多长时间？

大多数 CoddyKit 课程大约需要 5–10 分钟。每节课都很精短且互动，所以你能稳步进步，并在网页和应用中从离开的地方继续。

我能在这节 Apache Kafka & Stream Processing Fundamentals 课中编写并运行代码吗？

能。每节 Apache Kafka & Stream Processing Fundamentals 课都包含内置代码编辑器，你可以在浏览器中直接编写并运行真实代码，并获得即时 AI 反馈 — 无需本地设置。

此课程中的所有课时

← 返回 Apache Kafka & Stream Processing Fundamentals