I'm wanting to know how Kafka would handle this situation. A consumer has come across a poison pill message, and is not committing past it. No one notices for a long time (15 days). The retention period on the topic is (7 days). Let's say that this poison pill is in a log segment file that has satisfied the requirements to be deleted by the retention period.
What happens?
- Does Kafka allow this log segment file to be deleted while a Consumer actively trying to read from it?
- Does Kafka delete the log segment file and leave the Consumer scrambling trying to figure out where to start reading from by using the
auto.offset.reset
setting?