We are trying non blocking retry pattern in our kafka kstream application using spring cloud stream kafka binder library with the below configuration for the retry topics:
processDataRetry1:
applicationId: process_demo_retry_1
configuration:
poll.ms: 60000
processDataRetry2:
applicationId: process_demo_retry_2
configuration:
poll.ms: 600000
We observed that after the stream processor(consumer) has started, the first message in the process_demo_retry_1 was consumed after the 60000ms (1min) and the first message in the process_demo_retry_2 topic was consumed after 600000ms (10mins). However the next messages are getting consumed immediately. poll.ms is only impacting the first message. Could you please provide some explanation on how poll.ms works for kafka streams? PS : We are trying to introduce a delay in consuming from the retry topics using poll.ms as the kafka documentation says poll.ms is the amount of time in milliseconds to block waiting for input.
