-1

I am having the 250MB data storing into redis Cache as a single hash object. I am using Spring RedisTemplate to read data from redis. But it's taking the the around 30 to 35 secs of time. 

    redisTemplate.opsForHash().put("masterMap","masterMap", masterMap);
    redisTemplate.opsForHash().get("masterMap","masterMap");

The requirement was to get data in milli secs. However it's taking 30 to 35 secs of time.How to read this much size of data very quickly from redis cache. Having any alternative ways to read data from redis or I have to change any configurations.

Can someone please guide me on this.

Manohar
  • 19
  • 6
  • 1
    Don't store single objects of that size. Every database will run into performance problems when you you read and write is a massiveblob. – Roland Weber Jun 27 '18 at 13:18
  • As per our application architecture, we cannot split this mastermap data into multiple objects. we have to store as a single object. Because of this we are facing this problem. – Manohar Jun 27 '18 at 13:37

1 Answers1

0

Do a profiler run.

  • If you spend most of the time deserializing this data, then consider faster serialization methods. Like protobuf or cap'n'proto.
  • If you spend most of the time reading massive amount of data from socket, then try to decrease the amount stored. Use compression or/and normalization. For example, if there are a lot of strings with low cardinality, store their dictionary as a separate structure.
Imaskar
  • 2,773
  • 24
  • 35
  • I see that you use spring, so I bet for the 1st option. – Imaskar Jun 28 '18 at 14:03
  • Thanks for your reply, I have tried 2nd approach is compression data object, It's also not giving the good result in my case. – Manohar Jun 29 '18 at 09:33
  • @Manohar what is the output of `time redis-cli hgetall masterMap` ? I mean, you need to compare getting this map via spring and via minimalistic interface. – Imaskar Jun 29 '18 at 09:42
  • Wait a minute, you use `opsForCache` to store a map? How it translates to Redis keys? If you store it as a whole and retrieve it as a whole, try to store it as a single value. Because right now it may spend some time trying to get/put each map entry separately. – Imaskar Jun 29 '18 at 09:45
  • When I run HGETALL masterMap on redis-cli,It's giving the 160 sec of time fro output. I am using opsForHash to store and retrieving of data. – Manohar Jun 29 '18 at 10:21
  • This is a very extreme use case. Do you need individual access to fields of this nap in redis? If no, pack it before storing. Use single String value, not a hash. It might save some time. And check if you're network-bound. Maybe you're passing too much data. – Imaskar Jun 29 '18 at 12:30