0

Is it possible for Flink's CEP library to emit the data per input received? even if there is not any pattern matched? for example suppose there is this pattern

pattern: ba
input stream: a b a a
expect output stream is: F F T F

the default behavior is like this:

_ _ T _ 
_ = times that there is not any output.

the simplest solution that comes to my mind is left joining the input with output stream (there is not any left join on streams in flink and I should prepare it by coFlatMap) and mapping to the output (chang Nones with the value and drop the input) but I do not know if is it a good solution (performance-wise) or not.

Mohammad Reza Esmaeilzadeh
  • 1,313
  • 18
  • 31

1 Answers1

0

You could use union [1] to combine the input (or a transformed version of the input) with the result coming from CEP. Note that for this to work, all of the DataStreams being unioned must have the same type.

[1] https://nightlies.apache.org/flink/flink-docs-release-1.15/docs/dev/datastream/operators/overview/#union

David Anderson
  • 39,434
  • 4
  • 33
  • 60
  • Thanks, I want to have an output per input (not more or less) for example if there is a signal I want to return T and otherwise I want to return F, I don't think the union can help to create that output except there is some way to remove the duplicated events (one with F that is generated from input and T that is generated from pattern detection output.) – Mohammad Reza Esmaeilzadeh Sep 20 '22 at 10:35
  • 1
    Then I'll suggest you do a join (as you proposed). The easiest way to get this done will be to use MATCH_RECOGNIZE to do the pattern matching, and then do a SQL join. – David Anderson Sep 21 '22 at 00:11