Federation Plugin for Clustering
Explore the Federation plugin to connect multiple RabbitMQ brokers into a logical cluster without complex networking. Create distributed messaging systems that span data centers.
Federation Plugin for Clustering is a free RabbitMQ Messaging & Async Systems lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the RabbitMQ Messaging & Async Systems learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
What is RabbitMQ Federation?
Imagine you have RabbitMQ brokers in different data centers or cloud regions. How do they share messages without complex network setups?
The Federation plugin allows you to loosely connect these brokers. It enables messages to flow between them, creating a distributed messaging system across geographical boundaries.
Federation vs. Traditional Clustering
It's important to distinguish Federation from traditional RabbitMQ clustering:
- Clustering: Tightly couples nodes, sharing state and data. Best for high availability within a single, low-latency network (e.g., a data center).
- Federation: Loosely connects brokers by replicating messages. Ideal for spanning wide area networks (WANs) or connecting brokers managed by different teams. It doesn't share state like a cluster.
Key Components: Upstreams & Policies
Federation relies on two main concepts:
- Upstreams: These define the source broker from which messages will be pulled. An upstream specifies the connection details (like URI) of the remote RabbitMQ instance.
- Policies: These rules determine which exchanges or queues on your downstream broker should connect to which upstream broker. They use regular expressions to match names.
Enabling the Federation Plugin
Before you can use federation, you need to enable the plugin on all participating RabbitMQ brokers. This is done using the RabbitMQ command-line tool.
Run this command on each broker:
rabbitmq-plugins enable rabbitmq_federation rabbitmq_federation_managementConfiguring an Upstream Link
An 'upstream' tells your local broker where to pull messages from. You define it using the rabbitmqctl set_parameter command.
This example sets up an upstream named 'my-upstream' pointing to a remote broker:
rabbitmqctl set_parameter federation-upstream my-upstream \
'{"uri":"amqp://guest:guest@remote-host:5672","expires":3600000}'Creating a Federation Policy for Exchanges
Once an upstream is defined, you apply it to exchanges using a policy. This policy tells your local broker to federate messages for matching exchanges from the specified upstream.
This policy federates all exchanges starting with 'fed.' from 'my-upstream':
rabbitmqctl set_policy --apply-to exchanges \
fed-exchanges ".^fed\\..*" \
'{"federation-upstream":"my-upstream"}'How Federated Exchanges Work
With a federated exchange:
- A producer publishes messages to an exchange on the upstream broker.
- The federation plugin on the downstream broker pulls these messages from the upstream exchange.
- The messages then arrive at the corresponding federated exchange on the downstream broker, where local consumers can receive them.
The consumer only interacts with its local (downstream) broker.
Client Example: Consuming from Federated Exchange
Here's a Python consumer that connects to a downstream broker and receives messages from a federated exchange. It doesn't need to know the messages originated from an upstream broker.
To run this, ensure you have pika installed (pip install pika).
import pika
import sys
import os
def main():
# Connect to the local (downstream) RabbitMQ broker
connection = pika.BlockingConnection(pika.ConnectionParameters('localhost'))
channel = connection.channel()
# Declare the exchange that is federated from an upstream broker
# (e.g., using the policy created in previous steps for 'fed_logs')
channel.exchange_declare(exchange='fed_logs', exchange_type='fanout', durable=True)
result = channel.queue_declare(queue='', exclusive=True)
queue_name = result.method.queue
channel.queue_bind(exchange='fed_logs', queue=queue_name)
print(' [*] Waiting for federated messages. To exit press CTRL+C')
def callback(ch, method, properties, body):
print(f" [x] Received: {body.decode()}")
channel.basic_consume(queue=queue_name, on_message_callback=callback, auto_ack=True)
channel.start_consuming()
if __name__ == '__main__':
try:
main()
except KeyboardInterrupt:
print(' Interrupted')
try:
sys.exit(0)
except SystemExit:
os._exit(0)Federating Queues
You can also federate queues. When a queue is federated, messages published to that queue on the upstream broker will be pulled and added to the corresponding queue on the downstream broker.
The policy setup is very similar, just specify --apply-to queues:
rabbitmqctl set_policy --apply-to queues \
fed-queues ".^fed\\..*" \
'{"federation-upstream":"my-upstream"}'Quick Check on Federation
Which of the following best describes the primary use case for RabbitMQ Federation?
Recap: Federation for Distributed Messaging
In this lesson, you learned about the RabbitMQ Federation plugin:
- It allows you to connect RabbitMQ brokers across different locations.
- It uses upstreams to define source brokers and policies to apply federation rules.
- Federation differs from clustering by offering loose coupling, suitable for WANs.
- You can federate both exchanges and queues, enabling flexible message distribution in a distributed system.
This powerful plugin helps you build robust, geographically distributed messaging architectures.
Frequently asked questions
Is the “Federation Plugin for Clustering” lesson free?
Yes — the full text of “Federation Plugin for Clustering” is free to read here on the web, and the RabbitMQ Messaging & Async Systems course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the RabbitMQ Messaging & Async Systems course, upgrade to CoddyKit PRO.
What will I learn in “Federation Plugin for Clustering”?
Explore the Federation plugin to connect multiple RabbitMQ brokers into a logical cluster without complex networking. Create distributed messaging systems that span data centers. You practise RabbitMQ Messaging & Async Systems with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start RabbitMQ Messaging & Async Systems?
No prior experience is required. RabbitMQ Messaging & Async Systems on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Federation Plugin for Clustering” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this RabbitMQ Messaging & Async Systems lesson?
Yes. Every RabbitMQ Messaging & Async Systems lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Delayed Messages Plugin
- Shovel Plugin for Federation
- Federation Plugin for Clustering
- Message Deduplication & Consistent Hash Exchange Plugins