This kcat repo has an example of how to read the last N (=2000)
records in a Kafka topic (syslog
in this case):
$ kcat -C -b mybroker -t syslog -p 0 -o -2000 -e
The parameter -o
is offset from the end of the topic, and -e
indicates till the end of the topic. If I want to stream from one specific position Nth
to another position (N + k)th
, without knowing the full length of the topic, what would be the syntax?
I am trying to do some sort of random sanity-check on a data being produced where the data is too large to consume all. I wish to do this by randomly consuming a range of data and validate it. Hence, this question.