Linking two events in Flink based on a third event

Question

If two events stream into Flink, can they be logically joined using the information in a third event that follows them (using either the DataStream API or CEP)? E.g., can the third event in the example below be used to link the first two events based on its right_id and left_id?

ID: AAAA
ID: BBBB
ID: ZZZZ, right_id: AAAA, left_id: BBBB

score 1 · Accepted Answer · answered Dec 15 '16 at 19:18

This is a pretty basic CEP use case. The code would look something like this...

// data stream creation 
DataStream<Event> myStream = ... 

// cep pattern definition
Pattern<Event, ?> myPattern = Pattern.<Event>begin("first_event")
                .followedBy("second_event")
                .followedBy("match_event");

// cep pattern stream: apply pattern to stream
PatternStream<Event> myPatternStream = CEP.pattern(myStream, myPattern);

// create new data stream from pattern matches
DataStream<CEPEvent> myCEPEvent = myPatternStream.flatSelect(
                (Map<String, Event> pattern, Collector<CEPEvent> out) -> {

                // load potential event sequence matches
                Event first_event = pattern.get("first_event");
                Event second_event = pattern.get("second_event");
                Event match_event = pattern.get("match_event");

                // test event sequences 
                if (match_event.right_id.equals(first_event.ID)
                    && match_event.left_id.equals(second_event.ID)
                ){out.collect(new CEPEvent("successful cep hit"));}
            }
        );

Linking two events in Flink based on a third event

1 Answers1