EPL Every Limit Subexpression Lifetime

Question

I have a particular case for "every" pattern, this is the example code:

SELECT * FROM PATTERN [
        every(
         e1=Event(device_ip IN ( '10.10.10.1' ) AND category IS NOT NULL ))where timer:within(1800 Sec)
        ->              
         e2=Event(device_ip IN ( '10.10.10.1' ) AND event_cat_name.toLowerCase() IN ('User')  AND log_session_id = e1.log_session_id)
        ->
         e3=Event(device_ip IN ( '10.10.10.1' ) AND result.toLowerCase() IN ('Block') AND log_session_id = e1.log_session_id )
->
         e4=Event(device_ip IN ( '10.10.10.1' ) AND category IS NOT NULL AND log_session_id != e1.log_session_id)
        ->          
         e5=Event(device_ip IN ( '10.10.10.1' ) AND event_cat_name.toLowerCase() IN ('User')  AND log_session_id = e4.log_session_id)
        ->          
         e6=Event(device_ip IN ( '10.10.10.1' ) AND result.toLowerCase() IN ('Block') AND log_session_id = e4.log_session_id )
->
         e7=Event(device_ip IN ( '10.10.10.1' ) AND category IS NOT NULL AND log_session_id != e4.log_session_id)
        ->          
         e8=Event(device_ip IN ( '10.10.10.1' ) AND event_cat_name.toLowerCase() IN ('User')  AND log_session_id = e7.log_session_id)
        ->
         e9=Event(device_ip IN ( '10.10.10.1' ) AND result.toLowerCase() IN ('Block') AND log_session_id = e7.log_session_id )
];

Let me explain it:

We know that "every" pattern will reset after content were found and create another "Window" looking for same event, we are applying every pattern just to "e1" event in this case.

The above code is for 9 events, they are "grouped" by 3 events, as you see from "e1" to "e3" correspond to 1 unique event, from "e4" to "e6" another unique event and so on, each 3 events we know are same unique event because just those 3 events have same ID in "log_session_id", but we know that events from "e1" to "e3" are differents from "e4" to "e6" and different from "e7" to "e9" because every unique event has different ID in "log_session_id".

So when we have the following sequence of events: e1, e2, e3, e4, e5, e6, e7, e8, e9

When "e1" to "e3" are detected, the pattern every reset and search for the same event of "e1"... all is ok until now, but when "e4" event arrives, because of "e1" and "e4" are almost same conditions, the first window match and also match for the new window created when every restarted. We have in "e4" distint conditions but they are not known in "e1" positions yet, so because e1 does not know that e4 exists at first, in the new window is match and 2 Windows are opened until now: The first one have 2 unique events (e1 to e6) The second one have 1 unique window (e4 to e6).

And when "e7" to "e9" arrives, are generated another window, in total until now we have 3 Windows: The first one have 3 unique events (e1 to e9), The second one have 2 unique events (e4 to e9) The third one have 1 unique event (e7 to e9).

So, Do you know how to limit just to the first Window using every? we have tried with AND NOT but we cannot manage to do it.

score 0 · Answer 1 · answered Apr 08 '21 at 21:42

You could make that pattern much smaller and readable by using insert-into for the same-expression filters:

// use this for all the same-expression filters
insert into FilteredStream select Event(device_ip IN ( '10.10.10.1' )); 

select * from pattern [every(
         e1=FilteredStream(category IS NOT NULL )) where timer:within(1800 Sec)
        ....)];

For removing overlapping matches, there are multiple choices, such as:

Let the overlapping match happen and use a subquery on the output event to see if it overlaps
Use match-recognize instead which automatically skips-past-last match

EPL Every Limit Subexpression Lifetime

1 Answers1