0

Redis n00b here so don't shoot!

I need to store a collection of sets in redis indexed by key - I can:

import redis
r = redis.Redis()

r.sadd("A:B:123", *{456, 789})
r.sadd("A:B:124", *{666, 777})

but then if I want the collection "A:B" I have to collect the keys manually as in:

{k.rsplit(b':', 1)[-1]: r.smembers(k) for k in r.scan_iter("A:B:*") }
# {b'124': {b'666', b'777'}, b'123': {b'456', b'789'}}

This seems awfully slow (note also the rsplit)

I have been trying to use hmset to do the trick:

r.hmset("A:B", mapping={123: 'X', 124: 'Z'})

but I can't find a way of substituting my sets for 'X' and 'Z'.

(note that ideally set elements should be of type int as passed in - note also that those collections are meant to be read only, so I want to optimize lookup not insertion time)

Mr_and_Mrs_D
  • 32,208
  • 39
  • 178
  • 361

1 Answers1

2
import redis
r = redis.Redis()

r.sadd("A:B:123", *{456, 789})
r.sadd("A:B:124", *{666, 777})

New code

import redis
r = redis.Redis()

r. hmset("A:B", {"123": ",".join(map(str, {456, 789}))})
r. hmset("A:B", {"124": ",".join(map(str, {666, 777}))})

Print All Elements

print(dict([(k,set(map(int, v.split(b",")))) for k,v in r.hgetall("A:B").items()]))

Use a single map to store related keys and in a given key store concatenated values as a string.

NOTE: Adding element(s) to set is not atomic since a procedure has to read first then perform deserialization and store it back unless LUA script is used.

Using SUNION

r.sadd("A:B:123", *{456, 789})
r.sadd( "A:B:members", "A:B:123")
r.sadd("A:B:124", *{666, 777})
r.sadd( "A:B:members", "A:B:124")

Print Method

r.sunion(r.smembers("A:B:members"))
sonus21
  • 5,178
  • 2
  • 23
  • 48
  • Thanks! but is there no way of retrieving the whole dict at once? The sets are big and reconstructing the dict on every query is not an option. I knew I could store strings but I what I want is to get the sets back as stored - not do string operations that are super slow for my needs – Mr_and_Mrs_D Jan 20 '21 at 19:04
  • Which version of Redis you have? – sonus21 Jan 20 '21 at 19:12
  • Using https://github.com/andymccurdy/redis-py 3.5.1 – Mr_and_Mrs_D Jan 20 '21 at 19:23
  • see the edited answer, you can use SUNION if you're using Redis 6.2+ – sonus21 Jan 20 '21 at 19:31
  • Yes thanks tried - what this essentially adds is storing the keys itself in a set so I avoid scanning for them - so the code in OP becomes `dict((k, r.smembers(k)) for k in r.smembers("A:B:members"))` -> `{b'A:B:123': {b'789', b'456'}, b'A:B:124': {b'666', b'777'}}` - I still have to manually split the keys and reconstruct the collection – Mr_and_Mrs_D Jan 20 '21 at 19:35
  • That's correct there's so magical way to solve that using inbuilt redis commands. – sonus21 Jan 20 '21 at 19:37
  • I do not want the union of the sets - I want the dictionary mapping keys 123 etc to the sets. Same keys appear in other collections "A:Y", "Z:B" etc – Mr_and_Mrs_D Jan 20 '21 at 19:38