0

I am working on the apache beam streaming pipeline with Kafka producer as input and consumer for the output. Can anyone help me out with checkpoint in apache-beam

Akul Sharma
  • 69
  • 5
  • KafkaIO leverages Beam runner checkpointing mechanism and every runner has own implementation for this depending on data processing engine. Do you have any specific questions about that? – Alexey Romanenko Jul 10 '20 at 12:07
  • yes, I want to set State Backend to be FsStateBackend instead of MemoryStateBackend. I do not want to change the flink-conf.yaml in flink cluster as other jobs are also running on the same cluster, is there any other way to configure checkpointing URI and state backend. Please help me with that! – Akul Sharma Jul 13 '20 at 04:48

0 Answers0