I'm new to the streaming community I'm trying to create a continuous query using kafka topics and flink but I haven't found any examples so I can get an idea of how to get started can you help me with some examples? thank you.
Asked
Active
Viewed 132 times
-1
-
You'll find several examples here: https://docs.immerok.cloud/docs/how-to-guides/development/introduction/ (note, I work for immerok). – David Anderson Dec 10 '22 at 16:20
-
This example -- https://docs.immerok.cloud/docs/how-to-guides/development/batch-and-streaming-with-the-apache-flink-table-and-datastream-apis/ -- might be the best place to start. – David Anderson Dec 10 '22 at 16:22
-
You seem to be requesting outside sources (examples). Be advised that per our [on-topic guidance](https://stackoverflow.com/help/on-topic), requests for external resources (such as guides/examples and libraries, etc) are off-topic here. – Spevacus Dec 13 '22 at 18:13
1 Answers
1
For your use case, I'm guessing you want to use kafka as source for continuous data. In this case you can use kafka-source-connector(linked below) and if you want to slice it with time you can use flink's Window Processing Function. This will group your kafka messages streamed in a particular timeframe like a list/map.
Chandan Kumar
- 313
- 4
- 9
-
yes that's the idea, I'm trying to get sensor data as kafka source, and create a flink table then run continuous query over the table – user20740519 Dec 10 '22 at 19:44
-
@user20740519 you might find this link useful as a starting point https://github.com/redpanda-data/flink-kafka-examples – Chandan Kumar Dec 12 '22 at 10:07
-
thank you, but this one does not integrate topic data in a flink table – user20740519 Dec 12 '22 at 10:12
-
you might want to use rocksDB as Flink backend for this use case – Chandan Kumar Dec 12 '22 at 16:56
-
i dont want to store the data in a database just a flink dynamic table and to run a query over it continuously thanks – user20740519 Dec 12 '22 at 18:17