AI Agents · Lesson

Communication Protocols (Message Buses)

Decouple agents with a message bus (Redis, NATS, Kafka) so they can scale and fail independently.

Communication Protocols (Message Buses) is a free AI Agents lesson on CoddyKit — lesson 4 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the AI Agents learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.

Beyond In-Process Conversations

For small systems, agents call each other in-process. For larger systems, decouple them via a message bus — Redis, NATS, or Kafka.

Why Decouple?

Agents can scale independently
Failures isolated per agent
Async workflows (no need to wait for slow agents inline)
Replay messages for debugging
Cross-language: a Python agent and a Go agent can both subscribe

Simple Bus: Redis Pub/Sub

import redis
r = redis.Redis()

# Publisher
r.publish('agent.research.task', json.dumps({'task_id': 'abc', 'query': '...'}))

# Subscriber
p = r.pubsub()
p.subscribe('agent.research.task')
for msg in p.listen():
    if msg['type'] == 'message':
        handle_task(json.loads(msg['data']))

Pub/Sub for Broadcasting

Use when many agents may want to hear about an event (e.g. "user-question-received").

Queues for Work Distribution

Use a queue (Redis BLPOP, RabbitMQ) when exactly one worker should handle each task:

import json, queue

r = queue.Queue()

# Producer
r.put(json.dumps({'task_id': 'abc', 'kind': 'research'}))

def process(task):
    print('processing', task)

# Worker
while not r.empty():
    raw = r.get()
    task = json.loads(raw)
    process(task)

NATS for Speed

NATS is a lightweight message broker designed for microservices. Sub-millisecond latency, request/reply built in:

import nats
nc = await nats.connect('nats://localhost:4222')

await nc.publish('agent.research', json.dumps(task).encode())

# Request/reply
response = await nc.request('agent.research', payload, timeout=10)

Kafka for Durability

Kafka adds durable, replayable logs. Perfect for audit and replay:

from confluent_kafka import Producer, Consumer

producer.produce('agent-events', key=task_id, value=json.dumps(task))
producer.flush()

Event Schemas

Define every message shape with Pydantic or Protobuf. Without schemas, your bus becomes a mess:

class ResearchTask(BaseModel):
    task_id: str
    user_id: str
    query: str
    deadline: datetime

class ResearchResult(BaseModel):
    task_id: str
    findings: list[str]
    duration_ms: int

Correlation IDs

Every related message carries the same correlation_id so you can trace a task across services:

record = {'correlation_id': 'abc', 'task_id': 'def', 'span': 'llm_call'}
print(record)

Idempotent Handlers

Messages may be delivered more than once (at-least-once semantics). Make handlers idempotent — same task_id is processed once.

Dead-Letter Queue

When a handler fails repeatedly, dump the message to a DLQ for human review instead of looping forever:

if attempts > MAX_RETRIES:
    r.rpush('queue:research:dlq', raw)
    log.error('Sent to DLQ', extra={'task_id': task_id})

Observability

Trace every message:

Trace IDs flow through the bus
Logs include task_id, correlation_id
Metrics: throughput, latency, error rate per topic

When to Adopt

Start simple — in-process agents. Move to a bus only when you have:

5+ agents in production
Need to scale agents independently
Workflows that span multiple services

At-Least-Once?

What does "at-least-once delivery" require of your handlers?

Recap

Decouple with Redis for simple cases, NATS for latency, Kafka for durability. Schemas, correlation IDs, idempotent handlers, DLQ.

Frequently asked questions

Is the “Communication Protocols (Message Buses)” lesson free?

Yes — the full text of “Communication Protocols (Message Buses)” is free to read here on the web, and the AI Agents course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the AI Agents course, upgrade to CoddyKit PRO.

What will I learn in “Communication Protocols (Message Buses)”?

Decouple agents with a message bus (Redis, NATS, Kafka) so they can scale and fail independently. You practise AI Agents with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.

Do I need any experience to start AI Agents?

No prior experience is required. AI Agents on CoddyKit is structured for beginners through advanced learners; this is — lesson 4 of 4, so you can start here or from the beginning and move at your own pace.

How long does the “Communication Protocols (Message Buses)” lesson take?

Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.

Can I write and run code in this AI Agents lesson?

Yes. Every AI Agents lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.

All lessons in this course

Conversation-Based Multi-Agent (AutoGen)
Hierarchical Supervisors (Orchestrator + Workers)
Agent Roles and Specialisations
Communication Protocols (Message Buses)

← Back to AI Agents