3

In cluster mode of Redis, is a piece of data with a specific key has to be stored in a specific node, no matter what data structure the it has (e.g. List/Hash)?

For example, I have a hash map:

HMSET website google www.google.com yahoo www.yahoo.com

The key of the hash map is "website", and the hash map has data {google:www.google.com, yahoo:www.yahoo.com}. In my understanding, the hash map is stored in only one node of the cluster. It will be not efficient when the hash map is large (e.g. 400M key-value pairs in one hash map).

My question is: is there a way to automatically distribute the contents of the hash map of the same key among the cluster? For example, store pair {google:www.google.com} in node 0 and store pair {yahoo www.yahoo.com} in node 1, when the key of the hash map is still "website"?

Archimedes Trajano
  • 35,625
  • 19
  • 175
  • 265
Menooker
  • 93
  • 1
  • 8
  • Don't use a hash. Just set simple string values that will be distributed evenly throughout your cluster. You'll lose the ability to easily iterate over the keyspace. – mwp Nov 11 '17 at 17:45

1 Answers1

3

In cluster mode of Redis, is a piece of data with a specific key has to be stored in a specific node, no matter what data structure the it has (e.g. List/Hash)?

Yes - every key is mapped to a hash slot, that a single cluster instance manages.

My question is: is there a way to automatically distribute the contents of the hash map of the same key among the cluster?

No - data is distributed between nodes at key level. A given key's data structure cannot be distributed between multiple shards. To distribute the data, you'll have to model it using more keys.

Correctly modeling your needs requires knowing what type of operations you'll be performing against your distributed "hash map" and their respective frequencies. Feel free to add this information to the question, or open a new one that is more focused on your requirements.

Itamar Haber
  • 47,336
  • 7
  • 91
  • 117