Kafka Consumer not reading messages

Question

I have Kafka v1.0.1 running on the single node and I am able to push the messages to the topic but somehow unable to consume the message from another node using the below python code.

from kafka import KafkaConsumer
consumer = KafkaConsumer(
    'kotak-test',
    bootstrap_servers=['kmblhdpedge:9092'],
    auto offset reset='earliest',
    enable auto commit=True,
    group id=' test1',
    value_deserializer-lambda x: loads (x.decode('utf-8')))

for message in consumer:
    message = message.value
    print (message)

I am constantly pushing the messages from the console using the below command:

bin/kafka-console-producer --zookeeper <zookeeper-node>:<port> --topic <topic_name>

and also I can read via console

please show producer configurations also. And also see the default group-id which kafka uses. Just checking is 'kmblhdpedge'' resolved ?. — abhilash_goyal, Jan 08 '19 at 10:55

score 0 · Answer 1 · answered Jan 08 '19 at 18:51

You're using the old Zookeeper based producer, but the newer Kafka based Consumer. The logic for how these work and store offsets are not the same.

You need to use --broker-list on the Console Producer

Similarly with Console Consumer, use --bootstrap-server, not --zookeeper

Also, these properties should not have spaces in them

auto offset reset='earliest',
enable auto commit=True,
group id=' test1',

Kafka Consumer not reading messages

1 Answers1