In Lettuce(4.x) for Redis how to reduce round trips and use output of one command as input for another command, especially for Georadius

Question

I have seen this pass results to another command in redis and using via command line this command works well :

src/redis-cli keys '*' | xargs src/redis-cli mget

However how can we achieve the same effect via Lettuce (i started trying out 4.0.2.Final)

Also a solution to this is particularly important in the following scenario :

Say we are using geolocation capabilities, and we add a set of locations of "my-location-category" using GEOADD

GEOADD "category-1" 8.6638775 49.5282537 "location-id:1" 8.3796281 48.9978127 "location-id:2" 8.665351 49.553302 "location-id:3"

Next, say we do a GeoRadius to get locations within 10 km radius of 8.6582361 49.5285495 for "category-1"

Now when we get "location-id:1" & "location-id:3"

Given that I already set values for above keys "location-id:1" & "location-id:3"

I want to pipe commands to do the GEORADIUS as well as do mget on all the matching results.

Does Redis provide feature to do that?

and / or how can we achieve this via the Lettuce client library without first manually iterating through results of GEORADIUS and then do manual mget.

That would be more efficient performance for the program that uses it.

Does anyone know how we can do this ?

Update This is the piped command for the scenario I discussed above :

src/redis-cli GEORADIUS "category-1" 8.6582361 49.5285495 10 km | xargs src/redis-cli mget

Now we need to know how to do this via Lettuce

Answer 1

IMPORTANT: never use KEYS, always use SCAN instead if you must.

This isn't really a question about Lettuce nor Java so I can actually answer it :)

What you're trying to do is use the results from a read operation (GEORADIUS) as input (key names) for another read operation (MGET). This type of flow can't be pipelined, well, just because of that - pipelining means that you don't need the answers for operations right away but in you case you do.

However.

Since you're reading String keys with MGET, you might as well just denormalize everything (remember, we're NoSQL) and store the contents of these keys in the Sorted Set's members, e.g.:

GEOADD "category-1" 8.6638775 49.5282537 "location-id:1:moredata:evenmoredata:{maybe a JSON document here}:orperhapsmsgpack"

This will allow you to get the locations and their "data" with one GEORADIUS call. Of course, any updates to location:1's data will need to be done across all categories.

A note about Lua scripts: while a Lua script could definitely save on the back and forth in this case, any such script will be against best practices/not cluster safe.

Answer 2

After digging around and studying Lua script, my conclusion is that removing round-trips in such a way can only be done via Lua scripts as suggested by Itamar Haber.

I ended up creating a lua script file (myscript.lua) as below

local locationKeys = redis.call('GEORADIUS', 'category-1', '8.6582361', '49.5285495', '10', 'km' ) 
if unpack(locationKeys) == nil then
    return nil
else
    return redis.call('MGET', unpack(locationKeys))
end

** of course we should be sending in parameters to this... this is just a poc :)

now you can execute it via command

src/redis-cli EVAL "$(cat myscript.lua)" 0

Then to reduce the network-overhead of sending across the entire script to Redis for execution, we have the option of registering the script with Redis.

Redis will give us a sha1 digested code for future references for that script, which can be used for next calls to that script.

This can be done as below :

src/redis-cli SCRIPT LOAD "$(cat myscript.lua)"

this should give back a sha1 code something like this : 49730aa2ed3034ee48f818e486tpbdf1b500b19e

next calls can be done using this code eg

src/redis-cli evalsha 49730aa2ed3034ee48f818e486b2bdf1b500b19e 0

The sad part however here is that the sha1 digest is remembered only so long as the instance of redis is running. If it is restarted, that the sha1 digest is lost. Then you do the SCRIPT LOAD once again. And if nothing changes in the script, then the sha1-digest code will be the same.

Ideally while using through client api, we should first attempt evalsha, if that returns a "No matching script" error, then as a fallback do script load, and procure the sha1 code once again, and create an internal map of that and use that sha1 code for further calls.

This can well be done via Lettuce. I could find the methods for those. Hope this gives a good insight into solution for the problem.