1

This question sounds very much like: this one, but I believe it is not. Whilst that question is very specific, I believe it doesn't provide enough to cover the doubts I have.

I am trying to set up a Redis Cluster for an application deployment I have. I use Redis to store various information like Session info, Scheduled Job meta-info etc. I have been using a single node instance thus far. However, I am thinking moving to a Redis Cluster for HA. I know that Redis is single threaded and only provides best effort consistency and is not a strong consistency provider. So as far as I am at a single node, I had no issues with consistency (except in terms of fault-tolerance). However when I move to a cluster setup this is still not true (at-least as per what I understand).

My questions are as follows:

  1. If I move to a Redis Cluster setup, do I compromise on consistency to gain HA? The Redis website itself says the cluster setup does not provide strong consistency guarantees given its asynchronous replication method. In that case what's the argument for people using/suggesting Redis to be a viable solution for storing sessions as in the previous post? Is it only true for a single node setup? Or is it that sessions are okay to have been lost once every whenever-it-happens?

  2. For Redis to be truly fault-tolerant we must use the persistence feature and if not it cannot re-generate state? (I believe this also comes with a slight compromise in performance)

  3. Am I correct in my understanding that Redis Cluster only provides HA in the sense the data is sharded and distributed and does not provide automatic fail-over? For which Redis Sentinel must be used?

  4. What other solutions do people use for fast-access data with strong consistency requirements?

Shabirmean
  • 2,341
  • 4
  • 21
  • 34

2 Answers2

2

I may not answer all the questions in depth. Before going into the details of your questions;

The relation between availability and consistency is not only Redis related but one of the core principals of distributed systems. It can be explained with CAP Theorem. Yes you will compromise consistency for high availability because you can't sacrifice partition tolerance in distributed systems. Some of the distributed database technologies provide configuration to have "strong" consistency with the tradeoff availability with quorum (such as Cassandra).

If you want HA then Redis cluster may not be what you are looking for. Redis Cluster is a good solution when you need to shard your data(distribute the load) across multiple nodes. It is "a must" when you reach the limits of the memory of your instance. What you may need is Redis Sentinel.

Redis Sentinel provides high availability for Redis. In practical terms this means that using Sentinel you can create a Redis deployment that resists without human intervention certain kinds of failures.

  1. The post you shared is almost 8 years old, it may not cover or answer all the requirement's of today. The post is not asking any scenarios or solutions to cover distributed Redis too.
  • Redis is still a great solution for sessions(perfect example for key/value). You may scale vertically and stay in one node to achieve strong consistency for sessions.
  • You may switch to some other database with configurable consistency(data accuracy) such as Cassandra and set your quorum according to the business needs. It will not be a silver bullet, there is always a tradeoff.
  • You may look for a third party tool for quorum or implement one to have strong consistency in Redis. Redis's quorum is different than Cassandra's.

The quorum is only used to detect the failure. In order to actually perform a failover, one of the Sentinels need to be elected leader for the failover and be authorized to proceed. This only happens with the vote of the majority of the Sentinel processes.

  1. Redis sentinel could be an answer here too. The official documentation covers a lot of details.

If a master is not working as expected, Sentinel can start a failover process where a replica is promoted to master, the other additional replicas are reconfigured to use the new master, and the applications using the Redis server are informed about the new address to use when connecting.

  1. Redis cluster's specifications and use cases are different than Sentinels. Redis Sentinels one of the most important power comes from leader election during failover. AFAIK, cluster doesn't have this(didn't try but saw some details in documentation).
  2. I indirectly answered and gave examples for this one. Vertical(Instead of horizontal) scaling could be an option. You may add more resources(RAM etc) to your instance. Another option could be considering Cassandra and make tuning for immediate consistency. The tradeoff is again availability. If your node(s) go(es) down, then both your reads and writes fail.
Ersoy
  • 8,816
  • 6
  • 34
  • 48
0

For fast-access data with strong consistency requirements - go with Cassandra. It's inherent quorum mechanism helps ensure consistency and the P2P architecture provides scalability with minimal configuration overhead

dipanjanb
  • 51
  • 3