1

We have redis cluster which holds more than 2 million and these keys has been updated with the time interval of 1 minute. Now we have a requirement to take the snapshot of the redis db in a particular interval For eg every 10 minute. This snapshot should not pause the redis command execution. Is there any async way of taking snapshot from redis ?

It would be really helpful if we get any suggestion on open source tools or frameworks.

Joshua
  • 442
  • 1
  • 5
  • 20

1 Answers1

2

The Redis BGSAVE is async and takes a snapshot.

It calls the fork() function of the OS. According to the Redis manual,

Fork() can be time consuming if the dataset is big, and may result in Redis to stop serving clients for some millisecond or even for one second if the dataset is very big and the CPU performance not great

Two million updates in one minutes, that is 30K+ QPS. So you really have to try it out, run the benchmark that similutes your business, then issue BGSAVE, monitor the I/O and CPU usage of your system, and see if there's a spike in your redis calling latency.

Then issue LASTSAVE, which will tell you when your last success snapshot happened. So you can adjust your backup schedule.

AwesomeHunter
  • 690
  • 6
  • 10
  • Currently we use this in our Staging environment to take snapshots . How ever we see lag in the redis sentinel deployment as the data is heavy. As you said we might need to try out further and find an optimal resource requirement for this query. – Joshua Aug 08 '20 at 09:26
  • 1
    30K+ QPS is actually quite high, so I think you might consider Redis Cluster. For example, three master nodes, so three snapshops taking at the same time, which will alleviate the CPU pressure a lot. Also try AOF and sync every second. It's not snapshot, but might be good enough. Other than this, maybe a better CPU or SSD, depends on the result from your benchmark. – AwesomeHunter Aug 08 '20 at 09:36