Kafka Producer stops my code

Question

I'm calling a function which sends some data from kafka producer, but after it sends I'm returning a response which doesn't return. The code gets stuck at return. Anyone any idea whats happening?

My code is as follows,

def postEvent(eventData):
    print("The eventData is...",eventData)
    timestamp = datetime.now().__format__("%Y-%m-%d %H:%M:%S")
    try:
        producer = KafkaProducer(bootstrap_servers=["host:port"])
        data = json.dumps(eventData).encode('utf-8')
        try:

            kafkaResponse = producer.send('streamTest', data)

            response ={'time': str(timestamp), 'kafkaResponse':kafkaResponse.get(), 
                       'postResult': 'true'}
            print('kafaka response is...', response)
        except ConnectionAbortedError:
                response ={'time': str(timestamp), 'postResult': 'false'}
        except kafka.errors.KafkaTimeoutError:
                response ={'time': str(timestamp), 'postResult': 'false'}
        print('kafaka response is...', response)
        return response
    except kafka.errors.NoBrokersAvailable:
        response = {'Response':'Kafka Errors... NoBrokersAvailable'}
        print('kafaka response ', response)
        return response

score 0 · Answer 1 · answered Oct 13 '17 at 19:18

It's unclear from your question which return statement it's hanging at.

I tested your code, and it worked perfectly for me with a Kafka 0.10.0.1 broker and kafka-python 1.3.5.

It's probably a Kafka cluster networking issue, so the two places you are likely to hang are either: 1. kafkaResponse.get() while you wait for the Future to resolve 2. No brokers available, while the brokers timeout. If you pass in multiple brokers, keep in mind that they will need to each timeout before it throws the NoBrokersAvailable error.

Kafka Producer stops my code

1 Answers1