How to get list of key values in the Redis database at once?

Question

Kind of new to Redis cache database. I have a scenario where I have a list of items (say around 5000) which on converting it to JSON format takes around 400 MB. So each time if any one of the items is modified, I have to get the value from the Redis database and have to change that particular item and again push it to the Redi database. This is kind of affecting the performance.

So instead of caching the whole items in one file. I have decided to cache the items individually in the Redis database. But If I need to get all the items at once, I have to loop and pass 5000 keys to Redis and get the data which will make 5000 calls in Redis. Is there any option to get all the items at once whenever necessary so that I can get all the data and also individual data wherever necessary? Also is there any limitations on how many keys can be stored in Redis?

Thanks in Advance

did you consider using redisjson.io? – Guy Korland Nov 21 '19 at 12:00 — Guy Korland, Nov 21 '19 at 12:00

score 1 · Answer 1 · answered Nov 21 '19 at 11:55

1

By using Hash in redis you can achieve this. Becoz by using HSET you can update the individual value, also by using HGETALL you can get all your 5000 data in single call

answered Nov 21 '19 at 11:55

Praga_t

532
3
15

score 1 · Accepted Answer · answered Nov 21 '19 at 17:37

You have many options here. Most likely the best is to use one hash to store all your items, one item per hash field. The item itself most likely has its own values, you can serialize each item in json. You can create with:

> hmset items 01 "{'name': 'myCoolItem', 'price': 0.99}" 02 "{'name': 'anotherItem', 'price': 2.99}" ...

You can get all items with:

> hgetall items
  1) "01"
  2) "{'name': 'myCoolItem', 'price': 0.99}"
  3) "02"
  4) "{'name': 'anotherItem', 'price': 2.99}" 
  ...

You can get a single item with:

> hget items 02
  "{'name': 'anotherItem', 'price': 2.99}"

This is a tradeoff between updating and getting items, assuming the update operation on a single item is not as frequent or the items don't have tons of fields. To update a given field of a given item, you still have to deserialize locally, modify, serialize and then update back to Redis, but you are doing it on a per-item basis.

If you have tons of fields per item or the update operation is rather frequent, you can use one hash key per item:

> hset item:01 name myCoolItem price 0.99
> hset item:02 name anotherItem price 2.99

This allows you to get, or modify, one field of one item at a time:

> hset item:02 price 3.49
> hget item:01 price
  "0.99"

You can then get all your items, using multiple hgetall commands. To avoid hitting the server once per item, you can use pipelining. See Is there MGET analog for Redis hashes?

A third option, you can consider using a module like RedisJson.

How to get list of key values in the Redis database at once?

2 Answers2