5

I'm evaluating the use of lua scrips in redis, and they seem to be a bit slow. I a benchmark as follows:

  1. For a non-lua version, I did a simple SET key_i val_i 1M times
  2. For a lua version, I did the same thing, but in a script: EVAL "SET KEYS[1] ARGV[1]" 1 key_i val_i

Testing on my laptop, the lua version is about 3x slower than the non-lua version. I understand that lua is a scripting language, not compiled, etc. etc. but this seems like a lot of performance overhead--is this normal?

Assuming this is indeed normal, are there any workaround? Is there a way to implement a script in a faster language, such as C (which redis is written in) to achieve better performance?

Edit: I am testing this using the go code located here: https://gist.github.com/ortutay/6c4a02dee0325a608941

Sam Lee
  • 9,913
  • 15
  • 48
  • 56
  • How *exactly* did you do both of these? How did you execute that loop in the non-Lua version? And how did you execute it in the Lua one? – Nicol Bolas Mar 22 '16 at 01:26
  • I am using a the Go library, and just looping a bunch of times. Here is my complete script for testing: https://gist.github.com/ortutay/6c4a02dee0325a608941 – Sam Lee Mar 22 '16 at 01:30

3 Answers3

10

The problem is not with Lua or Redis; it's with your expectations. You are compiling a script 1 million times. There is no reason to expect this to be fast.

The purpose of EVAL within Redis is not to execute a single command; you could do that yourself. The purpose is to do complex logic within Redis itself, on the server rather than on your local client. That is, instead of doing one set operation per-EVAL, you actually perform the entire series of 1 million sets within a single EVAL script, which will be executed by the Redis server itself.

I don't know much about Go, so I can't write the syntax for calling it. But I know what the Lua script would look like:

for i = 1, ARGV[1] do
  local key = "key:" .. tostring(i)
  redis.call('SET', key, i)
end

Put that in a Go string, then pass that to the appropriate call, with no key arguments and a single non-key argument that is the number of times to loop.

Nicol Bolas
  • 449,505
  • 63
  • 781
  • 982
  • Interesting. So your'e saying that a lot of the overhead is just compiling the script each time. I guess I expected it to keep a pre-compiled version around after the first run, or something like that. – Sam Lee Mar 22 '16 at 16:00
  • 1
    Redis only compiles the script once because the compiled version is cached, but if you want to run a big script it might be wise to use EVALSHA to save bandwidth. – maffews Apr 01 '17 at 03:13
  • 1
    EVALSHA saves not only bandwitdh! It saves computation of SHA over script body. SHA is rather expensive function even for small scripts, so it is always better to use EVALSHA . – funny_falcon Jul 03 '18 at 18:15
  • So what is the result? Interested in that :) I think lua version should be faster? – ch271828n Mar 03 '20 at 04:24
1

I stumbled on this thread and was also curious of the benchmark results. I wrote a quick Ruby script to compare them. The script does a simple "SET/GET" operation on the same key using different options.

require "redis"

def elapsed_time(name, &block)
  start = Time.now
  block.call
  puts "#{name} - elapsed time: #{(Time.now-start).round(3)}s"
end

iterations = 100000
redis_key = "test"

redis = Redis.new

elapsed_time "Scenario 1: From client" do
  iterations.times { |i|
    redis.set(redis_key, i.to_s)
    redis.get(redis_key)
  }
end

eval_script1 = <<-LUA
redis.call("SET", "#{redis_key}", ARGV[1])
return redis.call("GET", "#{redis_key}")
LUA

elapsed_time "Scenario 2: Using EVAL" do
  iterations.times { |i|
    redis.eval(eval_script1, [redis_key], [i.to_s])
  }
end

elapsed_time "Scenario 3: Using EVALSHA" do
  sha1 = redis.script "LOAD", eval_script1
  iterations.times { |i|
    redis.evalsha(sha1, [redis_key], [i.to_s])
  }
end

eval_script2 = <<-LUA
for i = 1,#{iterations} do
  redis.call("SET", "#{redis_key}", tostring(i))
  redis.call("GET", "#{redis_key}")
end
LUA

elapsed_time "Scenario 4: Inside EVALSHA" do
  sha1 = redis.script "LOAD", eval_script2
  redis.evalsha(sha1, [redis_key], [])
end

eval_script3 = <<-LUA
for i = 1,2*#{iterations} do
  redis.call("SET", "#{redis_key}", tostring(i))
  redis.call("GET", "#{redis_key}")
end
LUA

elapsed_time "Scenario 5: Inside EVALSHA with 2x the operations" do
  sha1 = redis.script "LOAD", eval_script3
  redis.evalsha(sha1, [redis_key], [])
en

I got the following results running on my Macbook pro

Scenario 1: From client - elapsed time: 11.498s
Scenario 2: Using EVAL - elapsed time: 6.616s
Scenario 3: Using EVALSHA - elapsed time: 6.518s
Scenario 4: Inside EVALSHA - elapsed time: 0.241s
Scenario 5: Inside EVALSHA with 2x the operations - elapsed time: 0.5s

In summary:

  • scenario 1 vs. scenario 2 show that the main contributor is the round trip time as scenario 1 makes 2 requests to Redis while scenario 2 only makes 1 and scenario 1 is ~2x the execution time
  • scenario 2 vs. scenario 3 shows that EVALSHA does provide some benefit and I am sure this benefit increases the more complex the script gets
  • scenario 4 vs scenario 5 shows the overhead of invoking the script is near minimal as we doubled the number of operations and saw a ~2x increase in execution time.
echappy
  • 533
  • 5
  • 13
0

so there is now a workaround using a module created by John Sully. It works for Redis and KeyDB and allows you to use the V8 JIT engine which runs complex scripts much faster than Lua scripts. https://github.com/JohnSully/ModJS

user12671031
  • 1