0

I have tested my service having one instance using SeekToCurrentErrorHandler, which is having one consumer in one consumer group. In case of failure, retry happens and the records get committed as I have user ACK mode as RECORD and did not use ACK mode as MANUAL, as that creates a problem.

Now, let us have 2 more instances of my service, which has the same consumer with same consumer group.

My question: Are these 2 new instances going to retry like the first instance do exactly the same if in case the records have been committed by the first instance?

  ContainerProperties containerProperties = factory.getContainerProperties();
  containerProperties.setAckOnError(false);
  containerProperties.setAckMode(AckMode.RECORD);
  containerProperties.setErrorHandler(new SeekToCurrentErrorHandler());

CONFIG

STCEH

Adrian Mole
  • 49,934
  • 160
  • 51
  • 83
nihar_m
  • 1
  • 1

1 Answers1

0

When you have multiple consumers in the same group, each consumer will get a subset of the partitions; each consumer is independent; there is no communication between the instances.

If consumer1 dies during retries and its partition(s) allocated to another consumer, the retries will start from the beginning.

There is no persistent data about retries, just in-memory for the current consumer.

Gary Russell
  • 166,535
  • 14
  • 146
  • 179
  • Thanks Gary. One question - I am using SeekToCurrentErrorHandler. Even i am not using any ACK mode still the offset is getting committed in case of any exception. Please can you help ? – nihar_m Jun 15 '20 at 15:51
  • The offset will not be committed until the retries are exhausted and the recoverer called. You can see all offset activity by setting the `commitLogLevel` container property to INFO. However, with older versions (before 2.3) the `ackOnError` property was `true` by default. It is now `false`. https://github.com/spring-projects/spring-kafka/issues/1027 If you are using an older version, set it to `false`. – Gary Russell Jun 15 '20 at 16:05
  • `ackOnError` is `false` for me and I have used `SeekToCurrentErrorHandler(new FixedBackoff(5000L,5L))`. Still the offset is getting committed after all retries exhausted. Why is it happening when I have not used any `AckMode` `RECORD` or `MANUAL` – nihar_m Jun 15 '20 at 16:18
  • `>Still the offset is getting committed after all retries exhausted.` That's what it is designed to do; you can avoid it by setting `ackAfterHandle` to `false` on the error handler, but it's pointless because if the next record in the the partition is successfully proceessed, the previous offset is implicitly committed. You would have to `stop(() -> { })` or `pause()` the container in the recoverer to prevent subsequent record processing. – Gary Russell Jun 15 '20 at 17:01