is it possible to call lua functions defined in other lua scripts in redis?

Question

I have tried to declare a function without the local keyword and then call that function from anther script but it gives me an error when I run the command.

test = function ()    
    return 'test'
end



# from some other script
test()

Edit:

I can't believe I still have no answer to this. I'll include more details of my setup.

I am using node with the redis-scripto package to load the scripts into redis. Here is an example.

var Scripto = require('redis-scripto');
var scriptManager = new Scripto(redis);

scriptManager.loadFromDir('./lua_scripts');

var keys    = [key1, key2];
var values  = [val];

scriptManager.run('run_function', keys, values, function(err, result) {
console.log(err, result)
})

And the lua scripts.

-- ./lua_scripts/dict_2_bulk.lua

-- turns a dictionary table into a bulk reply table
dict2bulk = function (dict)
    local result = {}
    for k, v in pairs(dict) do
        table.insert(result, k)
        table.insert(result, v)
    end
    return result
end


-- run_function.lua 

return dict2bulk({ test=1 })

Throws the following error.

[Error: ERR Error running script (call to f_d06f7fd783cc537d535ec59228a18f70fccde663): @enable_strict_lua:14: user_script:1: Script attempted to access unexisting global variable 'dict2bulk' ] undefined

Answer 1

I'm going to be contrary to the accepted answer, because the accepted answer is wrong.

While you can't explicitly define named functions, you can call any script that you can call with EVALSHA. More specifically, all of the Lua scripts that you have explicitly defined via SCRIPT LOAD or implicitly via EVAL are available in the global Lua namespace at f_<sha1 hash> (until/unless you call SCRIPT FLUSH), which you can call any time.

The problem that you run into is that the functions are defined as taking no arguments, and the KEYS and ARGV tables are actually globals. So if you want to be able to communicate between Lua scripts, you either need to mangle your KEYS and ARGV tables, or you need to use the standard Redis keyspace for communication between your functions.

127.0.0.1:6379> script load "return {KEYS[1], ARGV[1]}"
"d006f1a90249474274c76f5be725b8f5804a346b"
127.0.0.1:6379> eval "return f_d006f1a90249474274c76f5be725b8f5804a346b()" 1 "hello" "world"
1) "hello"
2) "world"
127.0.0.1:6379> eval "KEYS[1] = 'blah!'; return f_d006f1a90249474274c76f5be725b8f5804a346b()" 1 "hello" "world"
1) "blah!"
2) "world"
127.0.0.1:6379>

All of this said, this is in complete violation of spec, and is entirely possible to stop working in strange ways if you attempt to run this in a Redis cluster scenario.

Answer 2

Important Notice: See Josiah's answer below. My answer turns out to be wrong or at the least incomplete. Which makes me very happy ofcourse, it makes Redis all the more flexible.

My incorrect/incomplete answer:

I'm quite sure this is not possible. You are not allowed to use global variables (read the docs ), and the script itself gets a local and temporary scope by the Redis Lua engine.

Lua functions automatically set a 'writing' flag behind the scenes if they do any write action. This starts a transaction. If you cascade Lua calls, the bookkeeping in Redis would become very cumbersome, especially when the cascade is executed on a Redis slave. That's why EVAL and EVALSHA are intentionally not made available as valid Redis calls inside a Lua script. Same goes for calling an already 'loaded' Lua function which you are trying to do. What would happen if the slave is rebooted between the load of the first script and the exec of the second script?

What we do to overcome this limitation:

Don't use EVAL, only use SCRIPT LOAD and EVALSHA. Store the SHA1 inside a redis hash set.

We automated this in our versioning system, so a committed Lua script automatically gets it's SHA1 checksum stored in the Redis master, in a hash set, with a logical name. The clients can't use EVAL (on a slave; we disabled EVAL+LOAD in config). But the client can ask for the SHA1 for the next step. Almost all our Lua functions return a SHA1 for the next call.

Hope this helps, TW

Answer 3

Because I'm not one to leave well enough alone, I built a package that allows for simple internal calling semantics. The package (for Python) is available on GitHub.

Long story short, it uses ARGV as a call stack, translates KEYS/ARGV references to _KEYS and _ARGV, uses Redis as a name -> hash mapping internally, and translates CALL.<name>(<keys>, <argv>) to a table append + Redis lookup + Lua function call.

The METHOD.txt file describes what goes on, and all of the regular expressions I used to translate the Lua scripts are available in lua_call.py. Feel free to re-use my semantics.

The use of the function registry makes this very unlikely to work in Redis cluster or any other multi-shard setup, but for single-master applications, it should work for the foreseeable future.