1

The Redis INCR docs have some examples of using that command for a Rate Limiter based on IP addresses.

For instance, maintain a counter with a time-window as key, and INCR/EXPIRE it in a single transaction via MULTI/EXEC:

ts = CURRENT_UNIX_TIME()
keyname = ip+":"+ts
MULTI
    INCR(keyname)
    EXPIRE(keyname,10)
EXEC

A problem with this approach is having to maintain a time window in the counter key, and potentially having multiple keys for a single ip address. The docs then state:

An alternative implementation uses a single counter, but is a bit more complex to get it right without race conditions.

Why is it complex to get this right? The following implementation seems simpler than the former:

MULTI
    INCR(ip)
    EXPIRE(ip, 1, NX)
EXEC

The strategy here is simply to add NX, ensuring that EXPIRE only acts if there is no expiration yet. We also set the expiration to 1 second to match the desired rate limiting period. Since this is performed in a single transaction, we're ensured to call EXPIRE after INCR.

So, where is the race condition in this approach?

Note: There are already some questions about why a client sending 2 separate transactions (one for INCR and one for EXPIRE) would face a race condition (or, more precisely, drop before EXPIRE). This question is specifically for the case where both operations are performed in a single transaction via MULTI/EXEC.

Ken White
  • 123,280
  • 14
  • 225
  • 444
roim
  • 4,780
  • 2
  • 27
  • 35
  • I can see a race if the key expires between `INCR` and `EXPIRE`. In this case, the second `EXPIRE` technically isn't needed, but there are no negative effects from this that I can see. You could say that we should allow a request in this case (since the time window expired before our command finished), but that doesn't seem serious. – roim Oct 13 '21 at 01:33

1 Answers1

1

The NX option for EXPIRE command is added in Redis 7.0, which is still not released yet (2021/10/13). However, the Rate Limiter examples works for current version of Redis.

In fact, the Rate Limiter example also gives a Lua script solution:

local current
current = redis.call("incr",KEYS[1])
if current == 1 then
    redis.call("expire",KEYS[1],1)
end

This Lua script solution works as EXPIRE with NX option, and should be faster, since it saves round-trip-time.

for_stack
  • 21,012
  • 4
  • 35
  • 48
  • Oh, I was wondering why they suggested INCR+EXPIRE in one case and a lua script doing the same in another. NX being a newer addition explains it. Thank you! – roim Oct 13 '21 at 02:46
  • I'm not sure about saving round-trip time since the multi call is still a single request. I'm not sure if INCR+EXPR would be slower or faster than the LUA script. I guess if the script hash is sent as a string and not well compacted it could be close. – roim Oct 13 '21 at 02:47
  • By default, you send MULT, and wait for reply from server, then you send INCR, and wait for reply from server... That will cause multiple RTTs. However, if your client library supports pipelining the transaction commands, it will cost a single RTT. – for_stack Oct 13 '21 at 03:19
  • the docs state that MULTI is a single transaction: https://redis.io/topics/transactions Does that mean the Redis server will be blocked while RTs happen? Or does the server execute only once all RTs finish? Can't find this anywhere in the docs. (For my particular case I auto pipeline on the client, but still curious) – roim Oct 13 '21 at 03:52
  • 1
    Redis caches these commands, and runs these commands once it receives the `EXEC` command. When caching command, Redis is not blocked. – for_stack Oct 13 '21 at 04:37