0

I have developed a streamsets pipeline which using KAFKA consumer as origin.My pipeline is working fine if Kafka consumer having message in it.But kafka consumer have 0 message in it the my pipeline went into loop and running contineously and didn't finish.

I need to finish my pipeline if kafka consumer having zero messages in his topic.

Nishu Tayal
  • 20,106
  • 8
  • 49
  • 101
CMK
  • 40
  • 2
  • 10

1 Answers1

0

By default it is streaming application's nature to keep checking for messages. But a similar approach to what you want to achieve has been answered-here.

Tulika
  • 128
  • 8
  • The solution you mentioned above will work if at least one lag message in kafka topic . – CMK Feb 18 '22 at 15:07