Is it possible to re-read a message in Kafka Streams?

Asked Jul 03 '23 at 03:09

Active Jul 03 '23 at 05:33

Viewed 63 times

I'm processing a message from a topic in a Kafka stream in a transaction. Based on message processing logic I would like to be able to either move forward with a next message from the topic, or re-consume the currently processed message (sort of transaction rollback and retry initiated by the processing logic). Is this possible with Kafka Streams? Can throwing an exception during the message processing cause its re-consumption with Kafka stream?

The motivation for this question is to rate limit message processing in the stream, for example to make no more than one external API calls in 5 seconds, for the consumed messages.

edited Jul 03 '23 at 05:33

asked Jul 03 '23 at 03:09

AndCode

1

You could instead group data in Punctuator – OneCricketeer Jul 05 '23 at 05:24
1

Another option to @OneCricketeer 's answer is to cache the call's results in an expiring cache with a five minute expiry policy. Or if you want to get fancy, you can use an HTTP Connector. – Kacper Roszczyna Jul 06 '23 at 13:21

Is it possible to re-read a message in Kafka Streams?

0 Answers0