I'm reading from Kafka but the consumer hangs:
def produce(num, producer, topic_name):
print("hello2")
for i in range(num):
future = producer.send('my-topic', bytearray("Hello Kayla 00", "utf-8"))
try:
record_metadata = future.get(timeout=10)
except KafkaError:
log.exception()
pass
# Successful result returns assigned partition and offset
print (record_metadata.topic)
print (record_metadata.partition)
print (record_metadata.offset)
This is the outputs:
my-topic
0
256
my-topic
2
298
my-topic
1
299
my-topic
0
257
my-topic
0
258
So writing works perfectly fine. Furthermore, I have visibility with another tool on Kafka and I can see myself writing.
The consumer is where the problem is:
def consume(num, consumer):
print("WORK!!")
for message in consumer:
print("was here")
print(message.value)
if num == 0:
consumer.close()
break
The consumer never enters the for
loop. It just print WORK!!
and hands in the for message in consumer
forever.
Anyone knows what the problem might be?
This is how I setup the consumers and producers:
consumer = KafkaConsumer('my-topic',
group_id="Bayern-Munchen",
bootstrap_servers=bootstrap_servers,
auto_offset_reset='latest')
producer = KafkaProducer(bootstrap_servers=bootstrap_servers...)