0

Not getting expected behavior, my flink application getting live event and my trigger condition is depend on two event ABC and XYZ. when both event reach then trigger the notification.

application is using StreamTableEnviornment

here is the sql query that I am using

SELECT  * 
from EventTable 
where eventName in ('ABC','XYZ') 
and 1 IN (select 1 from EventTable where name='XYZ') 
and 1 IN (select 1 from EventTable where name='ABC')

use case: 1

ABC event comes -->nothing happens (as expected and waiting for XYZ event)

XYZ event comes --> condition match and sql query gives two event record(ABC &XYZ) and it trigger the notification (as expected)

Now again if I send 'ABC' event then sql query give the result ABC event and notification triggered.

I was expecting that query will not give result as only one event ABC reached and will wait for event XYZ. could you please help me with this behaviour? Am I missing something to get the expected result?

Ashutosh
  • 33
  • 8

1 Answers1

0

When the second ABC is added to the dynamic table, the first XYZ is already there, so the conditions are met. The addition of this third row to the input table causes one new row to be appended to the output table.

See Dynamic Tables in the documentation for more information about the model underlying stream SQL.

David Anderson
  • 39,434
  • 4
  • 33
  • 60
  • Thanks David, but I was thinking it follow the DAG graph and mark the condition that match and once all condition match it trigger. if it matches because XYZ is already there then why it gives only one record in result? – Ashutosh Aug 21 '20 at 04:42
  • 1
    The results are not being completely restated as each new row is added to the input. Just as the input arrives incrementally, the output is also an append stream. – David Anderson Aug 21 '20 at 07:44
  • Thanks, then is there any way to achieve my goal like give me result once ABC and XYZ reach. again give me result when ABC and XYZ reach( order not known and can come in any order)? if there is sql way/workaround that will be helpful. thanks again. – Ashutosh Aug 21 '20 at 08:03
  • I was exploring MATCH_RECOGNIZE but for this event order matters :( – Ashutosh Aug 21 '20 at 09:36
  • MATCH_RECOGNIZE includes a mechanism for sorting the events first. https://ci.apache.org/projects/flink/flink-docs-stable/dev/table/streaming/match_recognize.html#order-of-events – David Anderson Aug 21 '20 at 11:53
  • right, so order matter for MATCH_RECOGNIZE. if I say PATTERN (A B) and app event B is generated first and then A then it will not match the condition. :(. for my application order is unknown and thats why we didnt use CEP and was relying on SQL – Ashutosh Aug 21 '20 at 14:41
  • That makes sense. – David Anderson Aug 21 '20 at 15:21