1

I've got a large JSON array of objects stored in redis as an encoded string.

local string_foo = redis.call("get", "foo")

"[{\"id\":\"xxxxxxxx\",\"block-scope\":[],\"history\":[{\"type\":\"answer\",\"timestamp\":1516295540951,\"message\":{\"mid\":\"mid.$cAACRSqSkpgVnO4cWglhCkHOU0XJQ\",\"seq\":24216,\"text\":\"fdjl\"}},{\"messageType\":\"text\",\"type\":\"messa ..."

I'd like to iterate through this array using a lua script to migrate this data into a more manageable form. However, when I try to create a lua table using cjson decode...

local json_foo = cjson.decode(string_foo)

[[],[],[],[],[]...]

I get a list of empty arrays or sets (redis-cli ldb is unsure which)

1) (empty list or set)
2) (empty list or set)
3) (empty list or set)
4) (empty list or set)
5) (empty list or set)
....

Why is this happening? It's large, but not exceedingly so.(~6 MB) The string was encoded using JSON.stringify.

phillercode
  • 339
  • 1
  • 14

1 Answers1

2

If your JSON is an array of string/number/bool, you can just return json_foo, and Redis can parse the array for you.

However, your JSON is an array of object, which is too complicated for Redis to parse. You have to parse it in your Lua script. For example, you want to return all ids of your JSON array:

local json_foo = cjson.decode(string_foo)
local ids = {}
for idx, ele in pairs(json_foo) do ids[idx] = ele["id"] end
return ids
for_stack
  • 21,012
  • 4
  • 35
  • 48
  • This seems to be the answer, but it just displays the limitations of Lua scripting. This task was trivial to accomplish with a node script. – phillercode Aug 07 '18 at 08:43