42

Could you please explain me following example from "The Little Redis Book":

With the code above, we wouldn't be able to implement our own incr command since they are all executed together once exec is called. From code, we can't do:

redis.multi() 
current = redis.get('powerlevel') 
redis.set('powerlevel', current + 1) 
redis.exec()

That isn't how Redis transactions work. But, if we add a watch to powerlevel, we can do:

redis.watch('powerlevel') 
current = redis.get('powerlevel') 
redis.multi() 
redis.set('powerlevel', current + 1) 
redis.exec()

If another client changes the value of powerlevel after we've called watch on it, our transaction will fail. If no client changes the value, the set will work. We can execute this code in a loop until it works.

Why we can't execute increment in transaction that can't be interrupted by other command? Why we need to iterate instead and wait until nobody changes value before transaction starts?

Marboni
  • 2,399
  • 3
  • 25
  • 42
  • You are aware of the [incr](http://redis.io/commands/incr) comand in redis right? It does exactly what you want in your example, without using a transaction. Of course this is not an answer to the question itself, but nevertheless it is worth knowing. – polvoazul Sep 11 '13 at 16:53
  • 3
    @polvoazul, I know this command, thanks. It was a common question not caused by real case. – Marboni Sep 12 '13 at 05:01

1 Answers1

101

There are several questions here.

1) Why we can't execute increment in transaction that can't be interrupted by other command?

Please note first that Redis "transactions" are completely different than what most people think transactions are in classical DBMS.

# Does not work
redis.multi() 
current = redis.get('powerlevel') 
redis.set('powerlevel', current + 1) 
redis.exec()

You need to understand what is executed on server-side (in Redis), and what is executed on client-side (in your script). In the above code, the GET and SET commands will be executed on Redis side, but assignment to current and calculation of current + 1 are supposed to be executed on client side.

To guarantee atomicity, a MULTI/EXEC block delays the execution of Redis commands until the exec. So the client will only pile up the GET and SET commands in memory, and execute them in one shot and atomically in the end. Of course, the attempt to assign current to the result of GET and incrementation will occur well before. Actually the redis.get method will only return the string "QUEUED" to signal the command is delayed, and the incrementation will not work.

In MULTI/EXEC blocks you can only use commands whose parameters can be fully known before the begining of the block. You may want to read the documentation for more information.

2) Why we need to iterate instead and wait until nobody changes value before transaction starts?

This is an example of concurrent optimistic pattern.

If we used no WATCH/MULTI/EXEC, we would have a potential race condition:

# Initial arbitrary value
powerlevel = 10
session A: GET powerlevel -> 10
session B: GET powerlevel -> 10
session A: current = 10 + 1
session B: current = 10 + 1
session A: SET powerlevel 11
session B: SET powerlevel 11
# In the end we have 11 instead of 12 -> wrong

Now let's add a WATCH/MULTI/EXEC block. With a WATCH clause, the commands between MULTI and EXEC are executed only if the value has not changed.

# Initial arbitrary value
powerlevel = 10
session A: WATCH powerlevel
session B: WATCH powerlevel
session A: GET powerlevel -> 10
session B: GET powerlevel -> 10
session A: current = 10 + 1
session B: current = 10 + 1
session A: MULTI
session B: MULTI
session A: SET powerlevel 11 -> QUEUED
session B: SET powerlevel 11 -> QUEUED
session A: EXEC -> success! powerlevel is now 11
session B: EXEC -> failure, because powerlevel has changed and was watched
# In the end, we have 11, and session B knows it has to attempt the transaction again
# Hopefully, it will work fine this time.

So you do not have to iterate to wait until nobody changes the value, but rather to attempt the operation again and again until Redis is sure the values are consistent and signals it is successful.

In most cases, if the "transactions" are fast enough and the probability to have contention is low, the updates are very efficient. Now, if there is contention, some extra operations will have to be done for some "transactions" (due to the iteration and retries). But the data will always be consistent and no locking is required.

Didier Spezia
  • 70,911
  • 12
  • 189
  • 154
  • So, is there any "fairness" in this? What if I did the "watch" but it failed? Then I have to retry. What if there was contention - could I ever try forever without "getting the lock"? – Brad Sep 06 '13 at 21:18
  • So I think I understand from your example - that if you showed an example that did MULTI/EXEC but no WATCH, both sessions might read the value 10, and have Redis write back 11. It would would make the writeback "atomic", but wouldn't protect against the client reading and incrementing the same value locally, resulting in a wrong answer? – Brad Sep 06 '13 at 21:20
  • 1
    No fairness guarantee. However, in practice, having a client starving on a watch/multi/exec execution is not so common. – Didier Spezia Sep 09 '13 at 10:57
  • Second question: yes, exactly. – Didier Spezia Sep 09 '13 at 10:59
  • @DidierSpezia : Do we have to watch `powerlevel` again in session B? Or just repeat the transaction part? – cold_coder Oct 04 '17 at 09:08
  • You have to watch powerlevel again, because you have no guarantee that a third session is not messing with powerlevel at the same time. – Didier Spezia Oct 06 '17 at 14:51
  • @DidierSpezia Can I make WATCH after GET? I don't see a reason to start watching the key while reading the value – mtkachenko Jun 19 '18 at 14:52
  • @mtkachenko you have to WATCH before the GET otherwise you risk a lost update. Think of the scenario of two clients getting the value, one client pauses (maybe a GC pause who knows). The other client gets the watch and updates successfully. Then the paused client gets the same watch and updates successfully on the stale data he fetched BEFORE acquiring the WATCH. – Luke Kot-Zaniewski Aug 27 '21 at 11:29
  • @mtkachenko .. or even a simpler example requiring no arbitrary length pause: client 1 is already in the watch block about to update. Client 2 gets some T0 value. Client 1 updates value to T1 value and exits watch. Client 2 acquires watch and updates to T2 value completely ignoring what happened in T1. – Luke Kot-Zaniewski Aug 27 '21 at 11:43