Get or set cache for multiple keys in Redis with Python

Question

I'm trying to get and set multiple cache keys, but I need to update the cache if the key is not set yet, this is what I currently have:

import os
from redis.client import Redis

REDIS_HOST = os.environ["REDIS_HOST"]
REDIS_PORT = int(os.environ.get("REDIS_PORT", 6379))

redis_client = Redis(host=REDIS_HOST, port=REDIS_PORT, db=0)

DEFAULT_CACHE_TIMEOUT = int(os.environ.get("DEFAULT_CACHE_TIMEOUT", 60 * 60 * 24))


def get_or_set_cache(key, default, timeout=DEFAULT_CACHE_TIMEOUT, function_args=None, function_kwargs=None):
    # type: (str, any, int, *any, **any) -> any
    """Get value from cache or set default value to cache and return it"""
    cached_result = redis_client.get(key)
    if cached_result:
        return cached_result

    if type(default).__name__ == 'function':
        result = default(*function_args or (), **function_kwargs or {})
        redis_client.set(key, result, timeout)
        return result
    else:
        redis_client.set(key, default, timeout)
        return default

The problem with this is it is one key + value at the time, I would like to improve this by using similar logic to get/set multiple keys at the same time (1 connection to Redis), is there a way to do it?

This is what I tried:

Is there MGET analog for Redis hashes?

Most efficient way to get several hashes in Redis?

Python, redis: How do I set multiple key-value pairs at once

So it is possible to set or get multiple keys/values at the same time, but the missing logic is settting the cache if the key doesn't exists.

score 2 · Answer 1 · answered May 07 '23 at 10:48

What you might be looking for is the NX argument that you can use with SET command to implement "set the value, if the key does not exist".

As an example:

> SET NAME ANKIT NX
OK
> SET NAME SAHAY NX
(nil)
> GET NAME
"ANKIT"

As you can see, in the first command, along with SET I also pass NX which tells Redis to set NAME as ANKIT only when NAME does not exist already. In this case, since there is no such key, a key with NAME and value as ANKIT is created.

Next when I run the SET command on the same key NAME with NX argument, it does not change the key, becuase it already exists.

The same can be done for multiple keys using MSETNX as shown below:

> MSETNX key1 value1 key2 value2 key3 value3
(integer) 1

The only catch is, MSETNX will not set any keys if even one of them exists. For example, see the following:

> MSETNX key1 valuex key4 value4 key4 value5
(integer) 0

The above command will not create key4 and key5, just becuase one of the multiple keys key1 already exists.

score 0 · Answer 2 · answered May 07 '23 at 00:02

Yes use redis pipeline feature. pipeline = redis_client.pipeline() You can get the keys as pipeline.get(key) and then implement as per your use case. Design your logic base don what pipeline type is i.e. is it a dict or list or any other structure. Once you know that then creating logic is easy part. You see pipeline is dict so print it on stdout and visually see the data and then you know what else to do. Hope this helps.

Get or set cache for multiple keys in Redis with Python

2 Answers2