3

I have a Redis deployment acting as a caching layer. The cache is not very big and we would like to read all the items (key/values) at once from our Golang program. The only solution I found is to iterate through all master servers (it is a Redis cluster), execute a Scan to get all the keys and then iterate the keys to get the values.

err := cch.client.ForEachMaster(func(cl *redis.Client) error {
    wg.Add(1)
    defer wg.Done()
    var cursor uint64
    var n int
    numFor := 0
    for {

        var keys []string
        var err error
        keys, cursor, err = cl.Scan(cursor, "*", 10).Result()
        if err != nil {
            panic(err)
        }
        n += len(keys)

        for _, keyval := range keys {
            var cont Content
            if err := cch.items.Get(keyval, &cont); err == nil {

            } else {
                log.Warnf("Error getting Key %s from cache: %s from master %s", keyval, err)
            }
        }

        if cursor == 0 {
            break
        }
    }
    return nil
})
wg.Wait()

Is there a better way to do this? Cannot believe I need so many roundtrips to Redis to get the values. Thanks!

OCDev
  • 655
  • 1
  • 7
  • 21
  • You can reduce this to one round-trip per master by using EVAL to run a Lua script to do the same thing. Of course that will block the server while it's happening, but if it your data set is really so small then maybe that isn't a problem. – Kevin Christopher Henry Aug 28 '18 at 03:45
  • `SCAN` is the recommended way to do the job. Other options, e.g. `KEYS` will block Redis until all keys have been retrieved. Since you are using Redis Cluster, you might have many keys in Redis. If not use `SCAN`, you'll block Redis for a long time. – for_stack Aug 28 '18 at 11:18

1 Answers1

2

1) You can use KEYS command to get all the keys and then access every key. But that is not a recommended way in some cases since KEYS upon a big set of cache will cause long-blocking in Redis server. But if you just have a tiny cache, KEYS is simple and elegant command you can use.

2) You can also push the relevant keys into a hash using HSET command. So you can use HGETALL to get those key-values at once. This way can help your cache to become good-looking.

pfctgeorge
  • 698
  • 3
  • 9