0

I am trying to run Producer and Consumer within single function.

The flow is below :

message arrived --> consume from topic 1 --> transform --> produce to topic 2

If I instantiate Producer and consumer in this pattern (Producer first then consumer), it works fine.

def __init__(self, producer_config, consumer_config)
  self.producer = Producer(**producer_config)

  self.consumer_config = consumer_config
  self.consumer = Consumer(**consumer_config)
  self.consumer.subscribe([topic1])

But If I reverse the order and instantiate Consumer first then Producer,

def __init__(self, producer_config, consumer_config)    
  self.consumer_config = consumer_config
  self.consumer = Consumer(**consumer_config)
  self.consumer.subscribe([topic1])

  self.producer = Producer(**producer_config)

I get error :

{"level": "ERROR", "message": "Kafka consumer error: KafkaError{code=GROUP_AUTHORIZATION_FAILED,val=30,str="JoinGroup failed: Broker: Group authorization failed"}"}

I am just curious to know if this is working as intended and probable reasons for it.

dataplumber
  • 375
  • 3
  • 16

0 Answers0