0

In storm we execute collector.fail(tuple) to replay the tuple. What is the similar approach in Flink?

Raghavendar
  • 159
  • 1
  • 11

1 Answers1

0

Flink takes care of this automatically, so long as checkpointing is enabled. The underlying mechanism is described here, but in a nutshell, Flink periodically snapshots all of the cluster's state, and restores it in the event of a failure. This includes rewinding the sources and replaying events as needed to achieve exactly-once processing semantics.

Update:

Sometimes it is necessary to loop events back through some or all of the job graph. Flink's DataSet API supports this with iterations, but unfortunately, there are issues with iterations and streaming. The best available solution is to write events to a sink and re-ingest them from a source.

David Anderson
  • 39,434
  • 4
  • 33
  • 60
  • I have a specific business logic in flink function which is doing multiple API calls and these API calls might return some status code based on which I need to replay the same tuple. – Raghavendar Feb 23 '19 at 03:43