1

Below is how we are publishing messages to topic,

for(int i=0;i<=10000;i++){

  send("topic_name", "message");

}

We have stream listener as below for above topic,

   @StreamListener("topic_name")
    public void process(KStream<String, String> inputKStream) {
        //processing of messages
    }

Now the requirement is we need to stop processing of messages when API is triggered.

For that we have a controller as below, 

@Autowired
private CustomBindingEndpoint endpoint;

@PutMapping(value = "stop", produces = APPLICATION_JSON_UTF8_VALUE)
public void stopProcess() {
    endpoint.changeState("topic_name", State.STOPPED);
}

The CustomBindingEndpoint.java is same as https://github.com/spring-cloud/spring-cloud-stream/issues/1650#issuecomment-476626008

Once we trigger this API we expect the above StreamListener("topic_name") should get stop(further consuming has to be stopped)

But once we trigger, even though binding.stop() is invoked for "topic_name" the consumer still consumes messages.

I understand that binding.stop() will stop the consumer from consuming message. Please correct if i am wrong. kindly suggest how we can achieve this.

Vijaya
  • 157
  • 1
  • 16
  • See my answer to [this question](https://stackoverflow.com/questions/58795176/stop-consume-message-for-stream-listener). – Gary Russell Dec 09 '19 at 15:14
  • Thank you for the quick response. followed https://stackoverflow.com/questions/58795176/stop-consume-message-for-stream-listener and https://github.com/spring-cloud/spring-cloud-stream/issues/1650 but consumers are not getting stopped.Could you please let me know what changes are required in order to stop consuming messages. – Vijaya Dec 12 '19 at 07:38
  • It's not clear what do you mean by `are not getting stopped.` Any stack trace? Any other diagnostic information you can provide? Unreproducible issue is usually not an issue. But may be you have a corner case which we have not thought of. In order for us to diagnose we need to be able to reproduce. So as I suggested in githib conversation, please create a small project where the issue is reproduced, push it to guthub and we'll look. – Oleg Zhurakousky Dec 12 '19 at 09:55
  • Please check edited question, added required details. – Vijaya Dec 12 '19 at 10:32
  • The consumer will stop when all the messages from the previous `poll()` are processed, not immediately. To stop immediately you must set the Kafka consumer property `max.poll.records` to 1 (but that will degrade performance). – Gary Russell Dec 12 '19 at 13:15
  • @GaryRussell, it is not required to stop immediately.Ex: message1,2,3..10 got started processing(listener's got invoked) in between binding.stop() got triggered. Then from message1 - 10 can get process next processing should be stopped. – Vijaya Dec 12 '19 at 13:39
  • The listener container calls `poll()`; let's say 10 records are returned. If you stop after #5, the container won't actually stop until all 10 have been processed. If that's not what you are seeing, edit the question to show the logs. – Gary Russell Dec 12 '19 at 13:46
  • actually i just noticed inside stop() there is a null check for lifecycle, which is coming as null when i trigger binding.stop(). hope this is the reason consumers are not stopped.please correct me. – Vijaya Dec 12 '19 at 13:50
  • can we configure max.poll.records for @streamlistener? – Vijaya Dec 12 '19 at 14:14
  • Thanks @GaryRussell !!! It worked as expected when I changed input binding target from KStream to SubscribableChannel. – Vijaya Dec 13 '19 at 06:43
  • Once binding got stopped, if we restart application the messages are getting consumed automatically. Is there any way to purge messages from topic programmatically?. Please let me know. – Vijaya Dec 13 '19 at 09:47

0 Answers0