Rack awareness e distribuzione multi-AZ

Rack awareness tells Kafka about the physical (or cloud) topology of your brokers.

Without it, all replicas of a partition could land on brokers sharing the same rack or availability zone. If that zone fails, the partition becomes unavailable.

• Each broker advertises a broker.rack value.
• Kafka then places replicas on different racks when possible.

Set broker.rack in each broker's server.properties to identify its rack or zone.

When a topic is created, Kafka's replica assignment algorithm alternates across racks.

For a partition with replication factor 3 and 3 racks, each replica lands on a distinct rack — guaranteeing the partition survives the loss of any single rack.

In AWS, GCP, or Azure, map broker.rack to the availability zone (AZ).

• AZs are physically isolated within a region.
• Spreading replicas across AZs protects against a full AZ outage.
• Beware cross-AZ network costs for replication traffic.

No special flag is needed — if brokers have broker.rack set, topic creation is automatically rack-aware.

Use --describe to inspect which brokers hold each replica, then cross-check against broker racks.

Rack awareness protects placement, but durability also depends on min.insync.replicas.

With replicas spread across 3 AZs and min.insync.replicas=2, a producer using acks=all can still write even if one entire AZ is down.

Kafka 2.4+ supports fetch from follower. Consumers can read from a replica in their own rack, cutting cross-AZ traffic and latency.

Rack awareness is best-effort:

• If racks < replication factor, some replicas share a rack.
• Reassignments triggered manually do not always respect racks unless you generate a rack-aware plan.
• It does not rebalance existing topics when you add broker.rack later.

To fix existing topics after enabling rack awareness, generate a reassignment plan with the reassign-partitions tool.

Before going to production:

• Replication factor >= number of racks you want to tolerate losing + 1.
• Set min.insync.replicas to at least 2.
• Use acks=all on critical producers.
• Enable follower fetching to control cross-AZ costs.

Test your understanding of rack awareness.

You learned how rack awareness spreads replicas across racks and availability zones.

• Set broker.rack per broker.
• Topic creation becomes automatically rack-aware.
• Combine with min.insync.replicas and acks=all for AZ-failure resilience.
• Use follower fetching to reduce cross-AZ cost.