0

Maybe I am missing something, this seems too simple. Is it possible to make redis durable by having a master redis node duplicate data to a slave redis node?

My situation I have a REST endpoint which upon recieving a request from a client sticks the payload it in a redis queue and then returns a success (HTTP 200) to the client. If that queue goes down before the message is processed and before fsync occured, I've lost that payload and no one knows about it.

I was wondering, if instead I could simply write to two redis queues (in different zones) one the master and one the slave. When I write to the 'master' redis will then automatically write the same element in the slave queue and only then does the endpoint return a HTTP 200 to the client.

Is this possible? Redis would (i) need a way to write to a slave and (ii) have a synchronous API or awaitable API which will only return once there is confirmation the payload has been written to both the master and slave. The key here is that redis allows the caller to know that the slave has received the event.

If the client doesn't get a HTTP 200 they know they should try sending it again. Feel like there are caveats I'm not seeing.

Thanks

friartuck
  • 2,954
  • 4
  • 33
  • 67

1 Answers1

0

Is this possible?

Short answer. NO, it's NOT possible.

Redis would (i) need a way to write to a slave

Redis can replicate the data to slave. However, the replication is async, which means it will return response to client before the data has been written to slave.

(ii) have a synchronous API or awaitable API which will only return once there is confirmation the payload has been written to both the master and slave.

Since Redis 3.0, it supports WAIT command, which will block the client until the write operations of this client have been replicated to the given number of slaves.

This might mitigate the problem, at least you can ensure the the write operation have been replicated to serval nodes. However, you still might lose your data. Because the slave might also be down before it persists data to disk.

for_stack
  • 21,012
  • 4
  • 35
  • 48
  • "However, you still might lose your data. Because the slave might also be down before it persists data to disk.". You are saying that the both the master and slave might go down before being fsync is successful on both? I wonder... if I have more slaves to further mitigate the problem, can I WAIT on all of them to accept the write? Is that possible to use WAIT for that? – friartuck Aug 22 '21 at 04:17
  • YES, since fsync is async. Of course, you can have more slave to future mitigate the problem, but the performance might be bad. You might wait for a long time before returning HTTP 200 to your client. – for_stack Aug 22 '21 at 06:58