0

I have defined a @KafkaListener with following consumer properties. enable.auto.commit is by default true.

For one of the messages consumed, it threw an exception which was not caught anywhere. Now my console is getting filled with the same error again and again since the @KafkaListener is repeatedly consuming the same message. Two question about this:

1. To check my understanding, the message is read again and again because the message was not committed at that time because the consumer thread ended with the exception and the auto commit interval, 5 seconds, wasn't reached. And the listener always starts reading from the latest committed offset, so it again tries to read the failed msg.

Actually, I read the answer given by Gary Russell on this one, Spring Kafka Auto Commit Offset In Case of Failures. But didn't understand what he meant by

With auto-commit, the offset will be committed regardless of success/failure. The container won't commit after a failure, unless ackOnError is true (another reason to not use auto commit).

2. How can we avoid reading failed messages(meaning messages that threw off exception) again and again?

Consumer properties

dh1
  • 97
  • 1
  • 9

1 Answers1

0

Use of enable.auto.commit-true is not recommended; supported versions of the framework automatically set it to false (unless the user has explicitly set it to true).

It is not recommended because the listener container will commit the offsets in a more deterministic manner, using AckMode.BATCH (default) or AckMode.RECORD.

The redelivery of a failed record is performed by the default error handler (up to 9 retries with no back off, by default).

"failed record" means the listener threw an exception.

Gary Russell
  • 166,535
  • 14
  • 146
  • 179
  • The redelivery of a failed record is performed by the default error handler (up to 9 retries with no back off, by default). In my case the retries are happening continuously, even if I restart my application, it again starts to consume the failed message again – dh1 Jun 28 '23 at 15:08
  • 1
    As I said, `enable.auto.commit` is not recommended. – Gary Russell Jun 28 '23 at 15:14