Redis Cluster untuk Sharding

So far, we've learned about Redis replication and Sentinel for high availability. These are great for redundancy and read scaling, but what about write scaling or handling datasets larger than a single server's memory?

This is where Redis Cluster comes in. It's designed to provide horizontal scaling and fault tolerance by distributing your data across multiple Redis instances.

At its core, Redis Cluster uses sharding (also known as data partitioning). Imagine you have a massive library of books.

• Instead of one giant shelf, you split the books across many smaller shelves.
• Each shelf holds a portion of the books, and a librarian manages that shelf.

In Redis Cluster, each Redis instance (or node) acts like a 'librarian' managing a 'shelf' of your data. This allows you to scale both memory and CPU.

A Redis Cluster is a collection of interconnected Redis instances. Each instance can be a master or a replica.

• Master Nodes: These nodes hold and manage a portion of the dataset.
• Replica Nodes: These are copies of master nodes, providing high availability. If a master fails, one of its replicas can be promoted to take its place.

The cluster ensures data is distributed, and it can continue operating even if some nodes fail.

How does Redis know which node stores which piece of data? It uses hash slots.

• The entire dataset is divided into 16,384 logical slots (from 0 to 16383).
• Each master node in the cluster is responsible for a subset of these hash slots.
• When you store a key, Redis calculates a hash of the key to determine which slot it belongs to.
• This slot then maps to a specific master node.

This system allows for flexible data distribution and easy rebalancing.

To create a cluster, you need at least three master nodes for a fault-tolerant setup (each master can have replicas). Here's a conceptual CLI command using redis-cli:

redis-cli --cluster create 127.0.0.1:7000 127.0.0.1:7001 127.0.0.1:7002 --cluster-replicas 1

• This command creates a cluster with 3 masters (on ports 7000, 7001, 7002).
• --cluster-replicas 1 means each master will have one replica node.

The redis-cli tool automatically assigns hash slots to the master nodes.

When a client wants to interact with Redis Cluster, it can connect to any node. If the key the client requests doesn't belong to the connected node's hash slots, the node will respond with a MOVED redirection error.

Modern Redis client libraries (like Jedis for Java) are cluster-aware. They automatically handle these redirections by:

• Learning the cluster topology (which node owns which hash slots).
• Directly connecting to the correct node for a given key.

This makes interacting with a cluster surprisingly seamless for developers.

Let's see how a Java client (using the Jedis library) connects to a Redis Cluster. The client needs to know at least one node in the cluster to discover the full topology.

Try running this example:

One of the powerful features of Redis Cluster is its ability to reshard. This means you can dynamically add or remove nodes from the cluster while it's running.

• When you add new master nodes, you can migrate hash slots from existing masters to the new ones.
• When you remove nodes, their hash slots can be moved to other remaining masters.

This allows for flexible scaling up or down of your cluster resources without downtime.

Redis Cluster provides robust fault tolerance:

• If a master node fails, its assigned replica is automatically promoted to become the new master.
• The cluster continues to operate, as the hash slots previously managed by the failed master are now handled by its promoted replica.

However, if a master and all its replicas fail, the portion of the data managed by that master's hash slots becomes unavailable, and the cluster might cease to operate unless configured otherwise (cluster-require-full-coverage no).

In this lesson, we explored Redis Cluster, a powerful solution for scaling Redis horizontally. We learned:

• Cluster uses sharding to distribute data across multiple master nodes.
• Hash slots determine which node stores which key.
• Client libraries are cluster-aware, handling redirections automatically.
• It offers fault tolerance by promoting replicas upon master failure.
• The ability to reshard allows for dynamic scaling.

Redis Cluster is your go-to when a single Redis instance can no longer meet your application's data size or throughput demands.