flinkCEP query to database within select() function or SimpleCondition/IterativeCondition

Question

Using a JDB connector I make queries to a PostgreSQL database and process the results. This works fine!

BUT, when I try to query within a PatternFlatSelectFunction, in order the query results to be the condition on which I collect the new events, I get this error.

Can anyone help me? I would like to get a good understanding of what's going on.

Could you please not link images, but copy the code examples to the question itself. This way they won't get outdated. — Dawid Wysakowicz, Mar 04 '19 at 15:33

score 1 · Answer 1 · answered Mar 04 '19 at 15:38

The PatternFlatSelectFunction has to be Serializable as it is sent over to all TaskManagers.

If I understood you correctly you are trying to execute DataSet program from within the SelectFunction. That is a completely wrong approach and this is not possible, as that would mean you are spawning new jobs from within TaskManagers.

You may use other libraries to query your DB from within PatternSelectFunction. Also make sure it is Serializable.

flinkCEP query to database within select() function or SimpleCondition/IterativeCondition

1 Answers1