I am new to kafka. I want to understand how kafka consumer behaves when offset specified by it is not present. Maybe the message at the given offset is deleted due to retention policy or consumer specified an invalid offset value.
Asked
Active
Viewed 3,279 times
2 Answers
5
Your consumer will use your auto.offset.reset setting to reset its offset to a valid position. If no reset option has been specified, then the Broker will throw an InvalidOffsetException.
Simon Clark
- 624
- 3
- 11
-
1`auto.offset.reset=latest` is the default. See @sureshsiva's answer below. – arun Nov 21 '19 at 01:50
5
It depends on the auto.offset.reset's mode that you have configured. Please find below about what will happen with each mode when you seek to an offset that doesn't exist in the given topic's partition.
auto.offset.reset=earliest
- Partition's offset will be reset to 0 and all messages will be consumed
auto.offset.reset=latest
- Partition's offset will be reset to the last known offset and will be consuming only the new messages
auto.offset.reset=none
- Consumer application will fail with OffsetOutOfRangeException as shown below,
suresiva
- 3,126
- 1
- 17
- 23