3

Suppose that I have 2 different types of data streams, one providing weather data and the other providing vehicle data, and I would like to use Flink to do complex event processing on the data.

Which method in Flink 1.3.x is the correct method to use? I saw different methods like Union, Connect, Window Join. Basically I just want to try a simple CEP like this:

IF weather is wet AND vehicle speed > 60 
WITHIN the last 10 seconds
THEN raise alert

Thanks!

Jamie
  • 91
  • 1
  • 2
  • 6

1 Answers1

5

In my opinion, there are two ways how you can solve this problem:

  1. Use a common parent type for different types of events and connect two streams via union method before using CEP library.

  2. You can use flink-siddhi package to process the streams using SiddhiCEP what provides the way to describe a pattern (via SiddhiQL) for several data streams in the same time. More information about flink-siddhi here: https://haoch.github.io/flink-siddhi/. The source code is available on GitHub: https://github.com/haoch/flink-siddhi. Documentation for SiddhiCEP and SiddhiQL: https://docs.wso2.com/display/CEP420/SiddhiQL+Guide+3.1.

Hope, this information would be helpful.

Alex Chermenin
  • 838
  • 12
  • 24