mset over 400,000 map entries in redis

Question

I'm using java with lettuce redis client. I have a local Redis instance and I'm reading a file and put 400,000 entries into a hash map. But after the end of the mset process there were no keys/values added into the redis store. I realized the problem is with the size of the map entries.

It's working when the map size is less than 50,000 entries.

Is there a way to increase this size or should I send this map as chunks?

What is maximum memory allocated to redis ? – DhruvPathak Mar 19 '18 at 12:06 — DhruvPathak, Mar 19 '18 at 12:06
That line is commented. What should be the limit? – codebot Mar 19 '18 at 12:14 — codebot, Mar 19 '18 at 12:14

score 1 · Accepted Answer · answered Mar 19 '18 at 14:00

Sending thousands of items at once doesn't sound healthy. Encoding 800000 items requires a huge buffer. Split up MSET into multiple commands if you can (1000 seems a reasonable size to me as a general rule of thumb, better: benchmark it!) or use transactions if you require atomicity.

But after the end of the mset process there were no keys/values added into the redis store.

Did any errors occur or did was the command completed successfully? Checking debug logs of Redis and Lettuce might help.

mset over 400,000 map entries in redis

1 Answers1