2

I am using faust and asyncio to process mt kafka stream. I am also using aiomysql to create connection pool. When I start the worker, it process the messages from the stream but hangs after processing some of them. There are no error messages in the logs. Other tasks are running in the worker like some cron jobs. But the agent stops processing any further messages.

OneCricketeer
  • 179,855
  • 19
  • 132
  • 245
Rahul Hindocha
  • 170
  • 2
  • 14
  • Well, is the consumer at the end of the topic? – OneCricketeer Apr 07 '22 at 12:37
  • No. There is some lag. And it just keeps on increasing. I tried resetting the offset once. But after restarting, the application worked properly for some time but got hanged again. – Rahul Hindocha Apr 07 '22 at 12:46
  • Same behavior observeed. Changing topic config somehow restarts workers. – ixaxaar Jun 02 '23 at 08:10
  • Well, I figured out what was the issue with my consumer. In the end of the loop, I had written 'return' which made it exit the loop and not to consume any more messages from the stream. Make sure that the loop is ending with 'continue' – Rahul Hindocha Jun 11 '23 at 09:37

0 Answers0