0

This is my code.

    SplitStream<MonitoringEvent> splitStream =  inputStream.split(new OutputSelector<MonitoringEvent>() {

    @Override
    public Iterable<String> select(MonitoringEvent me) {

        List<String> ml = new ArrayList<String>();              
        ml.add(me.getEventType());                              
        return ml;
}

I have stream of Monitoring Events coming in random order temp : 80, pressure : 70 , humidity :80, temp:30...

With the above code, am splitting the stream , eventType wise i.e temperatureStream, pressureStream.

The problem is , if I know the eventType, i can select it from the splitStream like

splitStream.select('temperatureStream')

but the eventType is dynamic and not pre-defined.

How will I apply CEP for this dynamic stream. The CEP would be like, if the 

temperate is > 90 for past 10 minutes ...

pressure is > 90 for past 10 minutes ...
madhairsilence
  • 3,787
  • 2
  • 35
  • 76
  • Not ideal but since your eventTypes are finite and small (temp, pressure,humidity...), you can have multiple streams and then do type specific processing on those individual streams. If eventTypes grow significantly then yes it will be hard to manage. – Aurvoir Oct 25 '17 at 18:23
  • or split events upfront either at the source/producer or using some sort of key based routing like in messaging – Aurvoir Oct 25 '17 at 18:29
  • @madhairsilence any solution for this one? I have almost the same problem. – Joan Plepi Jul 13 '18 at 09:38
  • You cannot use CEP component of Flink for this. You will have to write your custom window event. And process it. Will try to post the code if possible. – madhairsilence Jul 16 '18 at 08:48

1 Answers1

0

Correct me if i'm wrong, but i think it isn't possible to do a dynamic lookup on select due flink's parallism. Your program gets translated into parallel instructions for flinks taskmanagers and the jobmanager coordinate these actions. Without overall knowlegde about your abstract syntax tree no parallism could be applied at all... Maybe you could find some in common attribute that all messages share and differ from there

lkaupp
  • 551
  • 1
  • 6
  • 17
  • So How do I achieve that? A single stream with multiple event types. And the stream had to be split and applied CEP accordingly – madhairsilence Mar 09 '17 at 08:42
  • If u didn't find a solution yet. You have to specifiy and parse the incoming message to classes using DeserializationSchema on your source. Here some speudo code e.g: class mySchema extends AbstractDeserializationSchema { public LogEvent deserialize(byte[] message) throws IOException {return (Temperature)message;} } with this every incoming message is Typed as Temperature Class und its possible to select each event by using .select().type(Temperature.class).dosomething with CEP-Rules – lkaupp Oct 25 '17 at 09:14