0Pricing
Advanced Spring Boot 4: Event-Driven Architecture (Kafka) · Lesson

Capacity Planning: Partitions and Replication

Learn how to size topics with the right number of partitions and replication factor to balance throughput, parallelism, durability, and operational cost.

Capacity Planning: Partitions and Replication is a free Advanced Spring Boot 4: Event-Driven Architecture (Kafka) lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Advanced Spring Boot 4: Event-Driven Architecture (Kafka) learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

Why Sizing Matters

Partition count and replication factor are hard to change after launch and directly shape throughput, parallelism, and durability. Good capacity planning avoids painful migrations later.

Partitions Drive Parallelism

A consumer group can have at most as many active consumers as there are partitions. More partitions allow more parallel consumers and higher throughput.

Estimating Partition Count

A common heuristic: divide target throughput by per-partition throughput.

  • Measure max producer throughput per partition (p).
  • Measure max consumer throughput per partition (c).
  • partitions = max(target/p, target/c).
partitions = ceil( targetMBs / min(producerMBs, consumerMBs) )

The Cost of Too Many Partitions

More partitions are not free:

  • More open file handles and memory on brokers.
  • Longer leader-election and rebalance times.
  • Higher end-to-end latency.

Plan headroom, but do not wildly over-provision.

Replication Factor

The replication factor is how many copies of each partition exist across brokers. A factor of 3 is standard for production durability.

In-Sync Replicas

With min.insync.replicas=2 and acks=all, a write succeeds only when at least two replicas confirm. This tolerates one broker failure without data loss.

min.insync.replicas=2
replication.factor=3

Creating a Sized Topic

Specify partitions and replication when creating a topic so it starts correctly sized.

kafka-topics.sh --create --topic orders \
  --partitions 12 --replication-factor 3 \
  --bootstrap-server localhost:9092

Planning for Growth

You can increase partitions later but never decrease them, and adding partitions breaks key-based ordering. Pick a count with growth headroom from the start.

Retention and Disk Sizing

Disk need = throughput x retention x replication factor. A topic at 10 MB/s with 7-day retention and RF 3 needs roughly 17 TB. Plan storage accordingly.

Spreading Across Brokers

Ensure partitions and replicas are evenly distributed so no broker becomes a hotspot. Rack-awareness places replicas across failure domains.

Putting It Together

Capacity planning balances parallelism, durability, latency, and cost. Size partitions from throughput, use RF 3 with min ISR 2, and plan disk from retention.

Quick Check

Test your understanding of capacity planning.

Recap

You learned capacity planning.

  • Partition count drives consumer parallelism and throughput.
  • Too many partitions add latency and broker overhead.
  • RF 3 with min ISR 2 tolerates one broker failure.
  • Plan disk from throughput x retention x replication.

Frequently asked questions

Is the “Capacity Planning: Partitions and Replication” lesson free?

Yes — the full text of “Capacity Planning: Partitions and Replication” is free to read here on the web, and the Advanced Spring Boot 4: Event-Driven Architecture (Kafka) course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Advanced Spring Boot 4: Event-Driven Architecture (Kafka) course, upgrade to CoddyKit PRO.

What will I learn in “Capacity Planning: Partitions and Replication”?

Learn how to size topics with the right number of partitions and replication factor to balance throughput, parallelism, durability, and operational cost. You practise Advanced Spring Boot 4: Event-Driven Architecture (Kafka) with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start Advanced Spring Boot 4: Event-Driven Architecture (Kafka)?

No prior experience is required. Advanced Spring Boot 4: Event-Driven Architecture (Kafka) on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “Capacity Planning: Partitions and Replication” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this Advanced Spring Boot 4: Event-Driven Architecture (Kafka) lesson?

Yes. Every Advanced Spring Boot 4: Event-Driven Architecture (Kafka) lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

  1. Performance Tuning Tips for Kafka
  2. Idempotent Producers and Consumers
  3. Deploying Spring Boot Kafka Apps to Cloud
  4. Capacity Planning: Partitions and Replication
← Back to Advanced Spring Boot 4: Event-Driven Architecture (Kafka)