FastAPI Backend Development Bootcamp · 课时

模式注册表与 Avro 契约演进

使用模式注册表强制执行事件契约，并依据兼容性规则演进载荷。

模式注册表与 Avro 契约演进 是 CoddyKit 上的免费 FastAPI Backend Development Bootcamp 课时。这是第 2 节课，共 4 节。你可以在下方免费阅读本课时的完整内容 — 然后在浏览器中使用内置代码编辑器和全天候 AI 导师进行实践。这是 FastAPI Backend Development Bootcamp 学习路径的一部分，你的进度在网页和 CoddyKit 应用中同步。 FastAPI Backend Development Bootcamp 课程共包含 4 节课。

本课时的部分内容尚未翻译，以英文显示。

Why Event Contracts Need a Registry

In an event-driven FastAPI backend, your service publishes events to Kafka or Pulsar and many independent consumers read them. The event payload is a contract: producers and consumers must agree on field names, types, and structure.

If the producer renames user_id to userId, every consumer breaks silently.
Plain JSON has no enforced shape, so a typo ships straight to production.

A Schema Registry stores versioned schemas centrally and rejects messages that violate the agreed contract, decoupling teams while keeping data safe.

Avro: A Compact, Schema-First Format

Apache Avro is the most common serialization format used with schema registries. Each record is described by a JSON schema, and the binary payload itself carries no field names, only values, making it compact.

The schema defines name, type, fields, and optional default values.
Readers need the schema to decode the bytes, which is exactly why the registry exists.

Below is a minimal Avro schema for an OrderCreated event.

order_created_schema = {
    "type": "record",
    "name": "OrderCreated",
    "namespace": "com.shop.events",
    "fields": [
        {"name": "order_id", "type": "string"},
        {"name": "user_id", "type": "string"},
        {"name": "amount_cents", "type": "long"},
        {"name": "currency", "type": "string"},
    ],
}

print(order_created_schema["name"], "has", len(order_created_schema["fields"]), "fields")

Serializing a Record with fastavro

The pure-Python fastavro library lets you encode and decode Avro records without any broker. This is the same byte layout your producer would push to Kafka.

parse_schema validates the schema once.
schemaless_writer writes the binary body; schemaless_reader decodes it.

Notice the encoded bytes contain values only, not field names.

import io
from fastavro import parse_schema, schemaless_writer, schemaless_reader

schema = parse_schema({
    "type": "record",
    "name": "OrderCreated",
    "fields": [
        {"name": "order_id", "type": "string"},
        {"name": "amount_cents", "type": "long"},
    ],
})

record = {"order_id": "o-123", "amount_cents": 4999}

buf = io.BytesIO()
schemaless_writer(buf, schema, record)
encoded = buf.getvalue()
print("encoded bytes:", encoded)

buf.seek(0)
decoded = schemaless_reader(buf, schema)
print("decoded:", decoded)

The Confluent Wire Format

When you publish through a registry, the value is not bare Avro. Confluent's serializer prepends a 5-byte header so consumers know which schema to fetch.

Byte 0: a magic byte, always 0x00.
Bytes 1-4: a big-endian 4-byte schema ID.
Remaining bytes: the schemaless Avro body.

The consumer reads the ID, downloads that exact schema version from the registry, and decodes the body. This is how old and new payloads coexist on the same topic.

import struct

MAGIC = 0
schema_id = 42
avro_body = b"\x0co-1234\x9eL"  # pretend Avro bytes

frame = struct.pack(">bI", MAGIC, schema_id) + avro_body
print("wire bytes:", frame)

magic, sid = struct.unpack(">bI", frame[:5])
print("magic:", magic, "schema_id:", sid)
print("body:", frame[5:])

Registering a Schema from FastAPI Startup

A clean pattern is to register your producer's schema once at application startup using the registry's REST API. The registry returns a stable schema ID you reuse for every message.

Subjects follow the <topic>-value convention by default.
Registering an identical schema is idempotent: you get the same ID back.

This snippet posts an Avro schema to a Confluent-compatible registry.

import json
import httpx

REGISTRY_URL = "http://schema-registry:8081"

async def register_schema(subject: str, avro_schema: dict) -> int:
    payload = {"schema": json.dumps(avro_schema)}
    async with httpx.AsyncClient() as client:
        resp = await client.post(
            f"{REGISTRY_URL}/subjects/{subject}/versions",
            json=payload,
            headers={"Content-Type": "application/vnd.schemaregistry.v1+json"},
        )
        resp.raise_for_status()
        return resp.json()["id"]

# Called inside FastAPI's lifespan startup:
# schema_id = await register_schema("orders-value", order_created_schema)

Compatibility Modes: The Core Decision

The registry enforces an evolution policy per subject. The mode you pick decides which schema changes are allowed and dictates your upgrade order.

BACKWARD (default): new schema can read data written by the previous schema. Upgrade consumers first.
FORWARD: previous schema can read data written by the new schema. Upgrade producers first.
FULL: both directions hold. Order does not matter.
*_TRANSITIVE: the check runs against all prior versions, not just the latest.

Most teams default to BACKWARD because consumers usually lag behind producers.

Backward-Compatible Change: Add a Field With a Default

Under BACKWARD compatibility, you may add a field only if it has a default. A consumer using the new schema reading an old message simply fills in the default; a removed field also needs the old one to have had a default.

Adding discount_cents with default: 0 is safe.
Adding it without a default is rejected, because old records have no value to supply.

The new version below evolves the order event safely.

order_v2 = {
    "type": "record",
    "name": "OrderCreated",
    "namespace": "com.shop.events",
    "fields": [
        {"name": "order_id", "type": "string"},
        {"name": "user_id", "type": "string"},
        {"name": "amount_cents", "type": "long"},
        {"name": "currency", "type": "string"},
        # NEW field is backward compatible ONLY because of the default
        {"name": "discount_cents", "type": "long", "default": 0},
    ],
}

print("fields in v2:", [f["name"] for f in order_v2["fields"]])

Reading Old Bytes With a New Schema

Avro resolves differences between the writer schema (used to encode) and the reader schema (used to decode). When the reader has a new field with a default, decoding old bytes injects that default automatically.

Pass both schemas to schemaless_reader as reader and writer.
The missing discount_cents appears as its default 0.

This is exactly what makes BACKWARD evolution non-breaking in production.

import io
from fastavro import parse_schema, schemaless_writer, schemaless_reader

writer = parse_schema({
    "type": "record", "name": "OrderCreated",
    "fields": [{"name": "order_id", "type": "string"},
               {"name": "amount_cents", "type": "long"}],
})
reader = parse_schema({
    "type": "record", "name": "OrderCreated",
    "fields": [{"name": "order_id", "type": "string"},
               {"name": "amount_cents", "type": "long"},
               {"name": "discount_cents", "type": "long", "default": 0}],
})

buf = io.BytesIO()
schemaless_writer(buf, writer, {"order_id": "o-9", "amount_cents": 1500})
buf.seek(0)
out = schemaless_reader(buf, writer, reader)
print(out)  # discount_cents filled from default

Breaking Changes the Registry Rejects

Some edits can never be compatible and the registry's compatibility check (a pre-flight POST .../compatibility/subjects/<s>/versions/latest) will return is_compatible: false.

Renaming a field (it becomes an add + a remove without aliases).
Changing a type incompatibly, e.g. string to long.
Adding a required field with no default under BACKWARD.

To rename safely, use Avro aliases so the reader maps the old name onto the new one.

# Safe rename using aliases: old name "user_id" -> new "customer_id"
renamed = {
    "type": "record",
    "name": "OrderCreated",
    "fields": [
        {"name": "order_id", "type": "string"},
        {
            "name": "customer_id",
            "type": "string",
            "aliases": ["user_id"],
        },
        {"name": "amount_cents", "type": "long"},
    ],
}

for f in renamed["fields"]:
    print(f["name"], f.get("aliases", []))

Checking Compatibility in CI Before Deploy

Catch breaking changes before they reach the broker by calling the registry's compatibility endpoint from your CI pipeline. If the proposed schema is incompatible, fail the build.

This protects every consumer without running a single message through Kafka.
Run it as a step in the same job that builds your FastAPI image.

The helper returns True only when the registry approves the new version.

import json
import httpx

REGISTRY_URL = "http://schema-registry:8081"

async def is_compatible(subject: str, new_schema: dict) -> bool:
    url = f"{REGISTRY_URL}/compatibility/subjects/{subject}/versions/latest"
    async with httpx.AsyncClient() as client:
        resp = await client.post(
            url,
            json={"schema": json.dumps(new_schema)},
            headers={"Content-Type": "application/vnd.schemaregistry.v1+json"},
        )
        resp.raise_for_status()
        return resp.json()["is_compatible"]

# In CI:
#   ok = await is_compatible("orders-value", order_v2)
#   if not ok: raise SystemExit("Schema change is incompatible")

Producing and Consuming With confluent-kafka

In production you let the serializer handle the wire format and registry lookups. confluent-kafka's AvroSerializer registers the schema, prepends the ID, and encodes the body; AvroDeserializer reverses it.

The serializer caches schema IDs, so the registry is hit rarely.
Consumers transparently fetch whatever writer schema each message was encoded with.

This is the glue between your FastAPI event publisher and downstream services.

from confluent_kafka.schema_registry import SchemaRegistryClient
from confluent_kafka.schema_registry.avro import AvroSerializer
from confluent_kafka import Producer

sr = SchemaRegistryClient({"url": "http://schema-registry:8081"})

schema_str = '''
{"type":"record","name":"OrderCreated",
 "fields":[{"name":"order_id","type":"string"},
          {"name":"amount_cents","type":"long"}]}
'''

serializer = AvroSerializer(sr, schema_str)
producer = Producer({"bootstrap.servers": "kafka:9092"})

# producer.produce(topic="orders",
#   value=serializer({"order_id": "o-1", "amount_cents": 999}, ctx))

Quick Check: Choosing a Compatibility Mode

Your team needs to add a new optional field to a Kafka event, and you cannot redeploy every consumer at the same moment as the producer. Consumers typically lag behind producers in deployment.

Recap: Contracts That Evolve Safely

You now know how to enforce and evolve event contracts in an event-driven FastAPI backend.

A Schema Registry stores versioned schemas and hands out a schema ID embedded in the Confluent wire format (magic byte + 4-byte ID + Avro body).
Avro separates writer and reader schemas, resolving differences via defaults and aliases.
BACKWARD (the common default) means new schemas read old data: add fields only with defaults, upgrade consumers first.
FORWARD upgrades producers first; FULL allows either order; TRANSITIVE variants check all prior versions.
Run the registry's compatibility check in CI to block breaking changes before they reach Kafka or Pulsar.

Treat your schemas as code: version them, review them, and let the registry guard the contract.

常见问题解答

「模式注册表与 Avro 契约演进」课时是免费的吗？

是的 — 「模式注册表与 Avro 契约演进」的完整文本可在网页上免费阅读。要进行交互式练习（内置代码编辑器和全天候 AI 导师）并解锁 FastAPI Backend Development Bootcamp 课程的其余内容，请升级到 CoddyKit PRO。 FastAPI Backend Development Bootcamp 课程共包含 4 节课。

「模式注册表与 Avro 契约演进」这节课中我会学到什么？

使用模式注册表强制执行事件契约，并依据兼容性规则演进载荷。你通过在浏览器中直接运行的动手代码来练习 FastAPI Backend Development Bootcamp，全天候 AI 导师会在你学习这节课的过程中回答你的问题。

学习 FastAPI Backend Development Bootcamp 需要有经验吗？

无需任何先前经验。CoddyKit 上的 FastAPI Backend Development Bootcamp 课程适合初学者到高级学习者，你可以从这里开始或从头开始，按照自己的节奏学习。这是第 2 节课，共 4 节。

「模式注册表与 Avro 契约演进」课时需要多长时间？

大多数 CoddyKit 课程大约需要 5–10 分钟。每节课都很精短且互动，所以你能稳步进步，并在网页和应用中从离开的地方继续。

我能在这节 FastAPI Backend Development Bootcamp 课中编写并运行代码吗？

能。每节 FastAPI Backend Development Bootcamp 课都包含内置代码编辑器，你可以在浏览器中直接编写并运行真实代码，并获得即时 AI 反馈 — 无需本地设置。

此课程中的所有课时

← 返回 FastAPI Backend Development Bootcamp