-2

I am using Jedis to connect to Redis and push data into a list. I am using rpush for the JSON data.

These are the steps I do:

  1. Fetch Data from Rabbitmq
  2. Collect info from JSON data and prepare a key , value pair
  3. Push the data into redis using the key and the value.

I dont see my code scaling more than 3000 requests per second.

Note:

I am not using pipeline , every message will result in getting jedis resource , add it to redis and close of resourse.

Vignesh Selvaraj
  • 1
  • I have built a cache in my code, where the messages from the rabbitmq are cached, if the cache size reaches 10k then I asynchronously add the data to redis. Now am able to see an increase of 5 to 6k messages per second – Vignesh Selvaraj Jul 03 '18 at 02:01
  • Performance varies a lot between different hardware and different tasks. How much is the size of your message? With lettuce I easily can get 40k/s on my local machine with single thread and about 120k with 4 threads. You probably want to know not 'how fast is Jedis', but 'where do I spend time?'. Use a java profiler for that. – Imaskar Jul 03 '18 at 07:46
  • My message is a json payload approximately 500 characters and persisting with a RPUSH. – Vignesh Selvaraj Jul 04 '18 at 10:44
  • You spend time on latency. You send a command and wait the result. If you'd send them in parallel, you could do much, much more. Also, try lettuce. It helps much with pipelining. – Imaskar Jul 04 '18 at 12:22

1 Answers1

-1

Options for persisting faster in Redis are

  1. Pipelining
  2. Jedis Connection Pooling

To Avoid: 3. No frequent opening/closing of resource, i.e open a resource and reuse it

Good link: https://tech.trivago.com/2017/01/25/learn-redis-the-hard-way-in-production/

How I solved My Problem: My design was perfectly fine. But I was pushing data into the same key for all my tests. When I started pushing data into different keys Performance increased hugely.

Vignesh Selvaraj
  • 1