kafka springboot about receiving messages only from consumer application launch time and ignoring unprocessed messages

Question

Currently when starting consumer application it will receive old messages that have not been processed by KafkaListener and I only want to receive the latest messages since starting the consumer application ignore those old messages, I have to do that any?

Answer 1

This is pretty brief introduction into your issue - it would be handy to show versions of libraries you are working with, configurations, etc.

Nevertheless, if you do not want to receive old messages, that has not been ack before, you need to move offset for you consumer group. Offset is basically pointer at last successfully read item, so when consumer is stopped, it remains here until consumer starts reading again - that is the reason why "old" messages are read. In this thread are some answers, but it is difficult to answer completely without further information.

Answer 2

Set consumer settings as auto.offset.reset=latest and enable.auto.commit=false, then your app will always start reading from the very end of the Kafka topic and ignore whatever was written while the app is stopped (between restarts, for example)

You could also add a random UUID to the group.id to ensure no other consumer would easily join that consumer group and "take away" events from your app.

Kafka Consumer API also has a method seekToEnd that you can try.