Шаблоны выбора лидера

In a distributed system, multiple instances of your application run simultaneously. Sometimes, you need one instance to perform a specific task, like processing a queue or coordinating updates, to avoid conflicts or duplicate work.

This is where leader election comes in. It's a process where distributed nodes agree on a single node to be the "leader" at any given time.

Redis is an excellent choice for implementing leader election due to its speed, atomic operations, and strong consistency guarantees for single-key operations.

• Atomic Operations: Commands like SETNX or SET ... NX EX execute entirely or not at all, preventing race conditions.
• Persistence: If configured, Redis can persist data, making leader election state more durable.
• Centralized State: Provides a single, agreed-upon source of truth for who the current leader is.

The simplest way to attempt leader election with Redis is using the SETNX command. SETNX key value ("Set if Not eXists") sets a key only if it doesn't already exist. If the key is set, it returns 1; otherwise, 0.

The first process to successfully set the leader key becomes the leader.

Consider the basic SETNX approach. What if the elected leader (process_A from the last example) crashes immediately after setting the LEADER_KEY, but before it has a chance to delete it?

The LEADER_KEY would remain in Redis indefinitely, preventing any other process from becoming leader. This creates a permanent deadlock, breaking your distributed system's high availability.

To prevent deadlocks, we must add an expiration time (Time-To-Live, or TTL) to the leader key. This ensures that even if a leader crashes, its leadership key will eventually expire, allowing a new election.

We can use the EXPIRE key seconds command right after SETNX to set a TTL.

While adding EXPIRE is better, a critical race condition still exists! Imagine this sequence:

1. Process A calls SETNX LEADER_KEY process_A. It succeeds (returns 1).
2. Process A then crashes before it can call EXPIRE LEADER_KEY 10.

The LEADER_KEY is set, but without a TTL, leading to the same deadlock situation as before. We need an atomic way to set the key and its expiration.

Redis provides a powerful, atomic SET command that combines setting a key's value and its expiration. The format is SET key value [EX seconds | PX milliseconds] [NX | XX].

• NX: Only set the key if it does not already exist (like SETNX).
• EX seconds: Set an expiration time in seconds.
• PX milliseconds: Set an expiration time in milliseconds.

Using SET LEADER_KEY MY_ID NX EX 10 ensures that both conditions (key not existing AND expiration) are applied atomically.

Once a leader is elected using the atomic SET command, it needs to periodically signal that it's still alive and capable of leading. This is done through "heartbeats."

A heartbeat involves the leader refreshing the expiration time of its leadership key before it expires (e.g., using EXPIRE LEADER_KEY NEW_TTL or PEXPIRE). If the leader fails to send heartbeats, its key will expire, triggering a new election among the remaining processes.

We've explored how Redis can facilitate leader election in distributed systems. We started with basic SETNX, identified its pitfalls, and learned to improve it with EXPIRE.

Crucially, we discovered the robust and atomic SET key value NX EX seconds command to prevent race conditions and ensure keys always have an expiration. Finally, we touched upon the importance of leader heartbeats to maintain active leadership.