2

I've been trying to implement retry logic for Spring cloud stream kafka such that if an exception is throw when producing an event to the topic sample-topic, It retries two more time.
I added in the following configuration to the application.properties file

spring.cloud.stream.bindings.processSampleEvent.destination=sample-topic
spring.cloud.stream.bindings.processSampleEvent.content-type=application/json
spring.cloud.stream.bindings.processSampleEvent.consumer.maxAttempts=2

I've written the lister code in way that it simply logs the received message and throws a NullPointerException so that I can test out the retry.

@StreamListener(ListenerBind.SAMPLE_CHANNEL)
  public void processSampleEvent(String productEventDto) {
    System.out.println("Entering listener: " + productEventDto);
    throw new NullPointerException();
}

But when testing out by producing an event to the sample-topic, I see that in the logs the event has been retries 20 times but I've specified in the properties to try only two time and also a weird thing happens when I change to it 3 times, It retries 30 times.
I'm pretty new to Spring cloud streams and any help on this would be really helpful. Thanks in Advance

Raj Kumar
  • 33
  • 1
  • 4

2 Answers2

1

The default error handler in the listener container is now a SeekToCurrentErrorHandler with 10 delivery attempts.

You can either disable the retries in the binder, and configure a STCEH with the retry semantics you want, or use retries in the binder and replace the default error handler with a simple LoggingErrorHandler.

To configure the container's error handler, add a ListenerContainerCustomizer<AbstractKafkaListenerContainerFactory> @Bean.

Gary Russell
  • 166,535
  • 14
  • 146
  • 179
  • 1
    I observed that the behavior of retrying 10 times of `maxAttempts` does not apply when `enableDlq=true`. Could you shed some light of the reason behind? – wltheng Aug 27 '21 at 01:53
  • Because the error is "recovered" by sending the record to the dead letter topic and no exception is thrown to the container, so its error handler is not invoked. – Gary Russell Aug 27 '21 at 12:28
  • I see, that makes sense. Nonetheless, isn't this Kafka binder's retry-10-times behavior contradicting with SCSt `maxAttempts` definition? Is there any plan to make them consistent? – wltheng Aug 28 '21 at 04:15
  • 1
    https://github.com/spring-cloud/spring-cloud-stream-binder-kafka/issues/1135 – Gary Russell Aug 30 '21 at 13:16
0

I faced the same problem. My working solution was to create a ListenerContainerCustomizer Bean, give it desired number of max attempts, and set consumer binding maxAttempts: 1

@Bean
public ListenerContainerCustomizer<AbstractMessageListenerContainer<?,?>> listenerContainerCustomizer(){
    return (container, dest, group) ->
          container.setErrorHandler(containerAwareErrorHandler());
}
public SeekToCurrentErrorHandler containerAwareErrorHandler(){
    return new SeekToCurrentErrorHandler(new FixedBackOff(0, maxAttempts-1);
}
Alexander Orlov
  • 33
  • 7