Guide me in writing aggregation (groupBy,orderBy) Lua script for Aerospike

Question

I have a following lua script that groups the data with 'sensorType' and print 'clientId' in each group of 'sensorType'.

function orderby(touples)
  local function mapper(rec)
  local element = map()
    element["clientId"] = rec["clientId"];
    element["sensorType"] = rec["sensorType"]
  return element
end

local function accumulate(currentList, nextElement)
   local sensorType = nextElement["sensorType"]
   local clientId = nextElement["clientId"]
     if currentList[sensorType] == nil then
         currentList[sensorType] = list()
     end

    list.append(currentList[sensorType],clientId)

  return currentList
end
local function mymerge(a, b)
   return list.merge(a, b)
end
local function reducer(this, that)
   return map.merge(this, that, mymerge)
end
return touples:map(mapper):aggregate(map{}, accumulate):reduce(reducer)
end

I also want groupBy with clientId like 'groupBy sensorType, clientId'. Please help me to prepare a script that can accept any number of columns for groupBy clause and do grouping with that.

currently my result is-

{ BEACON: [ 'client2', 'client2', 'client2', 'client2', 'client2', 'client2' ],
  SSID: 
   [ '100',
     '100',
     '100',
     '100',
     '100',
     '100',
     '100',
     '102',
     '100',
     '100',
     '101',
     '100' ] }

I want the result in this format -

{ BEACON: [ 'client2' ],
  SSID: 
   [ '100',
     '102',
     '101', ] }

Are you following this example: https://github.com/aerospike/orderby-example — lvolmar, Oct 20 '15 at 00:22

score 4 · Answer 1 · answered Oct 22 '15 at 20:22

In your function accumulate, clientId is unconditionally added to currentList. If you don't want redundant data in currentList, you need to check for membership of clientId within currentList.

This is a little tricky if you use a list rather than a set; you'll have to test each element individually:

local t = currentList[sensorType]
local alreadyInList = false
for i = 1, #t do
    if t[i] == clientId then
        alreadyInList = true
    end
end
if not alreadyInList then
    list.append(t, clientId)
end

This is rather slow--as currentList[sensorType] grows, it takes that much longer to test if it already contains an element you're trying to add. In many cases it won't make much of a difference, but using a set of elements instead of a list will be much faster (and easier). Sets in Lua are very easy since anything can be a key for a Lua table, even another table. Here's how you can use a table as a set instead of a list:

-- initialize a set as an empty table
if currentSet[sensorType] == nil then
    currentSet[sensorType] = {}
end
-- add the data to the set `currentList[sensorType]`
currentSet[sensorType][clientId] = true

-- when appropriate, convert the set back into a list
convertedList = {}
i = 0
for key in pairs(currentSet[sensorType]) do
    i = i + 1
    convertedList[i] = key
end

Before conversion, currentSet looks like:

{ [sensorType] =
    { ["100"] = true
    , ["101"] = true
    , ["102"] = true
    }
}

After conversion, convertedList looks like:

{ "100", "102", "101" }

(Note that convertedList could come out in any order, since the order of keys within a Lua table is undefined.)

Guide me in writing aggregation (groupBy,orderBy) Lua script for Aerospike

1 Answers1