1

I have a use-case where I need to fetch all the keys from AWS Elastic cache(redis), I thought of using multiGet to just avoid the network calls, but taking more time as compare to sending parallel requests below is the details

I am using springboot+jedis and I have 5000 keys in redis

  redisTemplate.opsForValue().get(key))
 // taking 100 ms 


return redisTemplate.opsForValue().multiGet(keys);
// All 5000 , taking around 24 second

Batches in to 500 and sending parallel using spring asyc task with 10 threads taking around ~5sec

return redisTemplate.opsForValue().multiGet(batchedkeys);

So I am not able understand what is the best way to fetch all the keys, as I read here that redis has single worker and multiple I/O threads. so single request with multi keys should have overall less response time as compare to multiple request with single keys due to reduced network calls. Pls correct me if my assumption is wrong

scoder
  • 2,451
  • 4
  • 30
  • 70
  • It seems there isn't anything wrong with your approach. Are you sure about your time calculation? May be time of other processes somehow got included? – sazzad Dec 14 '20 at 18:27
  • yes, my timestamps were at entry and exit of those calls . Added proper log also I checked the network logs – scoder Dec 15 '20 at 06:31
  • Did you try `redisTemplate.opsForValue().get(key))` for all keys? Are all of them within 100ms? – sazzad Dec 15 '20 at 13:54
  • Yes when I fetch all individually it was between 100 to 120 ms – scoder Dec 28 '20 at 06:10
  • I'm just spitballing here. Can you try using Jedis directly? – sazzad Dec 28 '20 at 06:55
  • I tried jedis directly but same.. I think though I am sending in multiget from app side, redis side it is looking in sequence (right from I/O to worker threads).. How to verify this? – scoder Jan 04 '21 at 13:19
  • MONITOR command – sazzad Jan 04 '21 at 14:09

0 Answers0