Apache Kafka & Stream Processing Fundamentals · 课时

单消息转换（SMT）

学习 Kafka Connect 的单消息转换如何在连接器与 Kafka 之间就地重塑记录，而无需编写流处理器。

单消息转换（SMT） 是 CoddyKit 上的免费 Apache Kafka & Stream Processing Fundamentals 课时。这是第 4 节课，共 4 节。你可以在下方免费阅读本课时的完整内容 — 然后在浏览器中使用内置代码编辑器和全天候 AI 导师进行实践。这是 Apache Kafka & Stream Processing Fundamentals 学习路径的一部分，你的进度在网页和 CoddyKit 应用中同步。 Apache Kafka & Stream Processing Fundamentals 课程共包含 4 节课。

本课时的部分内容尚未翻译，以英文显示。

What Are SMTs?

Single Message Transforms (SMTs) are lightweight functions applied to each record as it flows through a Kafka Connect connector.

They let you tweak data — rename fields, add metadata, route topics — without a separate processing job.

Where SMTs Run

SMTs run inside the connector worker:

For a source connector, after reading from the system and before writing to Kafka.
For a sink connector, after reading from Kafka and before writing to the target.

Configuring a Transform

Transforms are configured declaratively in the connector's JSON or properties. You name the chain, then configure each transform.

{
  "transforms": "addTs",
  "transforms.addTs.type":
    "org.apache.kafka.connect.transforms.InsertField$Value",
  "transforms.addTs.timestamp.field": "ingestedAt"
}

Common Built-in SMTs

Kafka Connect ships many useful transforms:

InsertField — add a static or metadata field.
ReplaceField — rename or drop fields.
MaskField — redact sensitive values.
ExtractField — pull one field up as the whole value.

Routing with RegexRouter

RegexRouter rewrites the destination topic name, handy for adding prefixes when ingesting many tables.

{
  "transforms": "route",
  "transforms.route.type":
    "org.apache.kafka.connect.transforms.RegexRouter",
  "transforms.route.regex": "(.*)",
  "transforms.route.replacement": "db_$1"
}

Chaining Transforms

List multiple transforms separated by commas; they run in order, each receiving the previous one's output.

{
  "transforms": "mask,route",
  "transforms.mask.type":
    "org.apache.kafka.connect.transforms.MaskField$Value",
  "transforms.mask.fields": "ssn,creditCard",
  "transforms.route.type":
    "org.apache.kafka.connect.transforms.RegexRouter",
  "transforms.route.regex": "(.*)",
  "transforms.route.replacement": "secure_$1"
}

Key vs. Value Variants

Many SMTs come in two forms, suffixed with $Key or $Value, so you can target the record's key or value independently.

For example InsertField$Key versus InsertField$Value.

Predicates

Predicates let a transform apply conditionally — for example, only to tombstone records or to a specific topic.

{
  "predicates": "isTomb",
  "predicates.isTomb.type":
    "org.apache.kafka.connect.transforms.predicates.RecordIsTombstone",
  "transforms.drop.predicate": "isTomb"
}

When NOT to Use SMTs

SMTs operate on one record at a time. They cannot:

Join across records or topics.
Aggregate or window.
Call external services efficiently.

For those, use Kafka Streams or ksqlDB instead.

Custom SMTs

You can implement your own by writing a class that implements the Transformation interface, packaging it as a JAR, and placing it on the worker plugin path.

public class UppercaseField
    implements Transformation<SourceRecord> {
  public SourceRecord apply(SourceRecord record) {
    // reshape and return the record
    return record;
  }
}

Best Practices

Use SMTs wisely:

Keep chains short and readable.
Mask sensitive data as early as possible.
Prefer predicates over per-topic connectors when filtering.
Move heavy logic to a stream processor.

Quick Check

Test your understanding of SMTs.

Recap

You learned about Single Message Transforms.

SMTs reshape each record inline within a connector.
Use built-ins like InsertField, ReplaceField, MaskField, RegexRouter.
Chain them and gate with predicates.
For joins, aggregations, or windows, use a stream processor instead.

常见问题解答

「单消息转换（SMT）」课时是免费的吗？

是的 — 「单消息转换（SMT）」的完整文本可在网页上免费阅读。要进行交互式练习（内置代码编辑器和全天候 AI 导师）并解锁 Apache Kafka & Stream Processing Fundamentals 课程的其余内容，请升级到 CoddyKit PRO。 Apache Kafka & Stream Processing Fundamentals 课程共包含 4 节课。

「单消息转换（SMT）」这节课中我会学到什么？

学习 Kafka Connect 的单消息转换如何在连接器与 Kafka 之间就地重塑记录，而无需编写流处理器。你通过在浏览器中直接运行的动手代码来练习 Apache Kafka & Stream Processing Fundamentals，全天候 AI 导师会在你学习这节课的过程中回答你的问题。

学习 Apache Kafka & Stream Processing Fundamentals 需要有经验吗？

无需任何先前经验。CoddyKit 上的 Apache Kafka & Stream Processing Fundamentals 课程适合初学者到高级学习者，你可以从这里开始或从头开始，按照自己的节奏学习。这是第 4 节课，共 4 节。

「单消息转换（SMT）」课时需要多长时间？

大多数 CoddyKit 课程大约需要 5–10 分钟。每节课都很精短且互动，所以你能稳步进步，并在网页和应用中从离开的地方继续。

我能在这节 Apache Kafka & Stream Processing Fundamentals 课中编写并运行代码吗？

能。每节 Apache Kafka & Stream Processing Fundamentals 课都包含内置代码编辑器，你可以在浏览器中直接编写并运行真实代码，并获得即时 AI 反馈 — 无需本地设置。

此课程中的所有课时

← 返回 Apache Kafka & Stream Processing Fundamentals