0

I'm using Siddhi to reduce the amount of events existing in a system. To do so, I declared a batch time window, that groupes all the events based on their target_ip. 

from Events#window.timeBatch(30 sec)
select id as meta_ID, Target_IP4 as target_ip
group by Target_IP4
insert into temp;

The result I would like to have is a single event for each target_ip and the meta_ID parameter value as the concatenation of the distinct events that forms the event.

The problem is that the previous query generates as many events as distinct meta_ID values. for example, I'm getting

  1. "id_10", "target_1"
  2. "id_11", "target_1"

And I would like to have

  1. "id_10,id_11", "target_1"

I'm aware that some aggregation method is missing in my query, I saw a lot of aggregation function in Siddhi, including the siddhi-execution-string extension which has the method str:concat, but I don't know how to use it to aggregate the meta_ID values. Any idea?

Community
  • 1
  • 1
Peter Rubi
  • 119
  • 1
  • 12

1 Answers1

0

You could write an execution plan as shown below, to achieve your requirement: 

define stream inputStream (id string, target string);

-- Query 1
from inputStream#window.timeBatch(30 sec)
select *
insert into temp;

-- Query 2
from temp#custom:aggregator(id, target) 
select *
insert into reducedStream;

Here, the custom:aggregator is the custom stream processor extension that you will have to implement. You can follow [1] when implementing it.

Let me explain a bit about how things work:

Query 1 generates a batch of events every 30 seconds. In other words, we use Query 1 for creating a batch of events.

So, at the end of every 30 second interval, the batch of events will be fed into the custom:aggregator stream processor. When an input is received to the stream processor, its process() method will be hit.

@Override
    protected void process(ComplexEventChunk<StreamEvent> streamEventChunk, Processor nextProcessor, StreamEventCloner streamEventCloner, ComplexEventPopulater complexEventPopulater) {
        //implement the aggregation & grouping logic here
}

The batch of events is there in the streamEventChunk. When implementing the process() method, you can iterate over the streamEventChunk and create one event per each destination. You will need to implement this logic in the process() method.

[1] https://docs.wso2.com/display/CEP420/Writing+a+Custom+Stream+Processor+Extension

Dilini
  • 777
  • 8
  • 22