Sending message to DLQ only for specific exception in Spring Boot binder Kafka

Question

I am using Spring Cloud Stream binder Kafka for message processing. As part of my requirement I need to send message to DLQ for only specific user defined exception and not for all RuntimeExceptions.

I followed this link and configured ListenerContainerCustomizer like below but still my code is sending all exceptions to DLQ. Also I tried to skip exception stacktrace in message header, even that is not happening. What am I missing here?

 @Bean
public ListenerContainerCustomizer<AbstractMessageListenerContainer<byte[], byte[]>> customizer(DefaultErrorHandler errorHandler) {
    return (container, dest, group) -> container.setCommonErrorHandler(errorHandler);
}
@Bean
public DefaultErrorHandler errorHandler(DeadLetterPublishingRecoverer recovered) {
    recovered.excludeHeader(DeadLetterPublishingRecoverer.HeaderNames.HeadersToAdd.EX_STACKTRACE);
    recovered.addNotRetryableExceptions(UserException.class);
    return new DefaultErrorHandler(recovered, new FixedBackOff(0L, 1L));
}
@Bean
public DeadLetterPublishingRecoverer publisher(KafkaTemplate<Object, SpecificRecord> kafkaOperations) {
    return new DeadLetterPublishingRecoverer(
            kafkaOperations,
            (consumerRecord, exception) -> {
                return new TopicPartition(dlqTopicName, 0);
            }
    );
}

You must disable retries and DLQ processing in the binder to delegate the work to the error handler (set `maxAttempts=1`). https://docs.spring.io/spring-cloud-stream/docs/current/reference/html/spring-cloud-stream.html#_retry_template — Gary Russell, Mar 06 '23 at 14:06
I have these 2 properties in application.yml "spring.cloud.stream.kafka.binder.consumer-properties.maxattempts=1 spring.cloud.stream.bindings.processin0.consumer.maxAttempts=1" — springenthusiast, Mar 06 '23 at 14:24

score 0 · Answer 1 · answered Mar 06 '23 at 14:54

Note that all records are sent to the recoverer, either immediately (for not retryable exceptions) or after retries are exhausted.

If you need different recovery behavior based on the exception type, you need a custom recoverer (i.e. delegate to the DLPR for some exceptions only).

This works as expected for me:

@SpringBootApplication
public class So75628212Application {

    public static void main(String[] args) {
        SpringApplication.run(So75628212Application.class, args);
    }

    @Bean
    Consumer<String> input() {
        return str -> {
            System.out.println(str);
            if (str.equals("foo")) {
                throw new IllegalStateException();
            }
            throw new IllegalArgumentException();
        };
    }

    @Bean
    ApplicationRunner runner(KafkaTemplate<byte[], byte[]> template) {
        return args -> {
            template.send("input-in-0", "foo".getBytes());
            template.send("input-in-0", "bar".getBytes());
        };
    }

    @Bean
    ListenerContainerCustomizer<AbstractMessageListenerContainer<byte[], byte[]>> cust() {
        return (container, group, dest) -> {
            DefaultErrorHandler eh = new DefaultErrorHandler((rec, ex) -> {
                System.out.println("Recovered: " + new String((byte[]) rec.value()));
            }, new FixedBackOff(0L, 1L));
            eh.addNotRetryableExceptions(IllegalStateException.class);
            container.setCommonErrorHandler(eh);
        };
    }

}

spring.cloud.stream.bindings.input-in-0.group=grp
spring.cloud.stream.bindings.input-in-0.consumer.max-attempts=1

Result:

foo
Recovered: foo
bar
bar
Recovered: bar

Sending message to DLQ only for specific exception in Spring Boot binder Kafka

1 Answers1