Flink CEP: FollowedBy pattern: same event loops the pattern multiple times

Question

I am new to Flink CEP and have been playing around with the patterns for better understanding of them. I have a simple case of a "begin" & a followedBy". I notice that in the case of followedBy, the same event is looping through it multiple times. What am I missing here?

Pattern match_win = Pattern.begin("first").where(new SimpleCondition() {

    public boolean filter(HitDTO hitDTO) throws Exception {
        boolean result = false;
        if (hitDTO.getHitScore() == 4)
        {
            System.out.println("First:" + hitDTO + ": " + hitDTO.getHitScore());
            result = true;
        }
        return result;
    }
}).followedBy("next").where(new SimpleCondition<HitDTO>(){

    public boolean filter(HitDTO hitDTO) throws Exception
    {
        boolean result = false;

        if (hitDTO.getHitScore() == 6)
        {
            System.out.println("Next:" + hitDTO+ ": " + hitDTO.getHitScore());
            result = true;
        }
        return result;
    }
});

I am passing in 4,4,6

Parallelism is set at 1.

StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment().setParallelism(1);

However, this is what I see in the logs for the printout within the patterns, where 6 is looping 4 times when it was passed in only once.

First:com.rs.dto.HitDTO@17619295: 4

First:com.rs.dto.HitDTO@c108f70: 4

Next:com.rs.dto.HitDTO@5d13aab8: 6

Just wondering why the same event is looping through multiple times but the outcome is correct.

Thanks for your answer.

score 0 · Answer 1 · answered Aug 28 '20 at 18:43

0

It is normal that the process of attempting to match a pattern to an input sequence will involve multiple evaluations of various components of the pattern. That's how pattern matching works: the pattern is compiled to a finite state machine, and all possible paths the input might take through that FSM are considered, looking for paths that lead to the terminal, matching state. If you're not careful in how you define your pattern, this can lead to a combinatorial explosion of effort.

answered Aug 28 '20 at 18:43

David Anderson

39,434
4
33
60

Thanks David for your answer. – Rishi Sarkar Aug 28 '20 at 19:18
I am passing 4 twice. I see it getting evaluated by "first" twice, great! Next when I pass in 6, it gets evaluated by "next" in the followedBy,which is fine. However, what puzzles me is why it gets evaluated 3 more additional times for that same run. Anything in my setup thats causing it? After its accepted, and exits the filter, it shouldn't be made to go through it again and again for THAT event, right? – Rishi Sarkar Aug 28 '20 at 19:29
Try replacing `.followedBy("next")` with `.next("next")` and compare what happens. – David Anderson Aug 28 '20 at 20:33
I was curious to understand the details better myself, so I rewrote this a bit to better reveal the walk that the pattern matcher is doing. https://gist.github.com/alpinegizmo/f51e924319f134373b59cb9814cf3db5 – David Anderson Aug 29 '20 at 09:53

Flink CEP: FollowedBy pattern: same event loops the pattern multiple times

1 Answers1