0

My typical use case is to keep monitoring an event stream to detect event patterns, but I expect the window is a hopping one. By default, interval ... in CEP SQL should define a tumbling window. Is it possible to have a hopping window for CEP pattern matching? or any other solutions?

Thanks

David Anderson
  • 39,434
  • 4
  • 33
  • 60
Grant
  • 500
  • 1
  • 5
  • 18

2 Answers2

3

It is possible to nest time-based queries in Flink SQL. So you can define a MATCH_RECOGNIZE clause on a view that did a windowing before.

Here is some sketch example:

-- get the rowtime from the window operation
CREATE TEMPORARY VIEW my_view AS
  SELECT HOP_ROWTIME(rowtime, INTERVAL '1' SECOND, INTERVAL '2' SECOND) AS windowedRowtime, ...
  FROM my_table
  GROUP BY HOP(rowtime, INTERVAL '1' SECOND, INTERVAL '2' SECOND);

-- use the new rowtime for MATCH_RECOGNIZE
SELECT * FROM my_view MATCH_RECOGNIZE(ORDER BY windowedRowtime ...)
twalthr
  • 2,584
  • 16
  • 15
  • In this case, I don't need to write `interval ..` in MATCH_RECOGNIZE, right? – Grant Mar 12 '21 at 23:55
  • Interval within a match recognize clause is a timeout to clean up stale state while waiting for the final end event. – twalthr Mar 15 '21 at 14:32
  • A further question, the created view only contains the aggregate values and group-by fields. what if i need all raw rows to feed into match_recognize? – Grant Mar 16 '21 at 19:54
1

If by "Flink CEP SQL" you mean MATCH_RECOGNIZE, then the WITHIN INTERVAL constraint doesn't really map onto any specific kind of windowing. You can use whichever AFTER MATCH strategy meets your needs.

David Anderson
  • 39,434
  • 4
  • 33
  • 60
  • i don't understand how `after match` can help in this case. can you give an example?Thx – Grant Mar 16 '21 at 19:58