0

I have a question geared towards using zendesk's deliveryboy/racecar wrappers in tandem for processing Kafka events.

I consume events using Racecar but send events back for reprocessing, at a delayed interval, into the same topic via DeliveryBoy using a Sidekiq scheduled job.

It worked a number of times but since, Racecar has stopped consuming those re-process events.

DeliveryBoy still confirms that it appended the message to the topic:

Sending 1 messages to b-4.sa-xxx.kafka.sa-east-1.amazonaws.com:9094 (node_id=4)
Opening connection to b-4.sa-xxx.kafka.sa-east-1.amazonaws.com:9094 with client id service_sidekiq...
Sending produce API request 1 to b-4.sa-xxx.kafka.sa-east-1.amazonaws.com:9094
Waiting for response 1 from b-4.sa-xxx.kafka.sa-east-1.amazonaws.com:9094
Received response 1 from b-4.sa-xxx.kafka.sa-east-1.amazonaws.com:9094
Successfully appended 1 messages to topic-normalized/7 on b-4.sa-xxx.kafka.sa-east-1.amazonaws.com:9094 (node_id=4)

but the consumer doesn’t pick up the event.

I know events written to the topic via other means are getting picked up by Racecar, seems to be the only ones its not consuming is the ones written by DeliveryBoy.

I’ve changed the client_id of DeliveryBoy to differ from the client_id of Racecar (write with one, listen with the other) but that didn’t help.

Any advice is appreciated.

mrlevitas
  • 80
  • 7
  • if the consumer was consuming messages earlier and now stopped working, there are two possible reasons - network failure or consumer group rebalance. If at the time of processing, another kafka consumer was brought up with same group id, it is possible that some of the events went to that kafka consumer. – Rishabh Sharma Aug 04 '20 at 21:51

1 Answers1

0

It ended up being that I had log level set incorrectly and was unable to witness side-effects since I was writing to debug.

Everything working as expected, user error..

mrlevitas
  • 80
  • 7