I am trying to run Producer and Consumer within single function.
The flow is below :
message arrived --> consume from topic 1 --> transform --> produce to topic 2
If I instantiate Producer and consumer in this pattern (Producer first then consumer), it works fine.
def __init__(self, producer_config, consumer_config)
self.producer = Producer(**producer_config)
self.consumer_config = consumer_config
self.consumer = Consumer(**consumer_config)
self.consumer.subscribe([topic1])
But If I reverse the order and instantiate Consumer first then Producer,
def __init__(self, producer_config, consumer_config)
self.consumer_config = consumer_config
self.consumer = Consumer(**consumer_config)
self.consumer.subscribe([topic1])
self.producer = Producer(**producer_config)
I get error :
{"level": "ERROR", "message": "Kafka consumer error: KafkaError{code=GROUP_AUTHORIZATION_FAILED,val=30,str="JoinGroup failed: Broker: Group authorization failed"}"}
I am just curious to know if this is working as intended and probable reasons for it.