Implement Kafka Producer Consumer Chat with Queue

Question

I have created implemented Kafka Producer-Consumer messaging with Topic using python.How can I do the same with Queue so that the message will be only devilered to a single consumer .

This is my Producer code

# Import KafkaProducer from Kafka library
from kafka import KafkaProducer

# Define server with port
bootstrap_servers = ['localhost:9092']

# Define topic name where the message will publish
topicName = 'First_Topic'

# Initialize producer variable
producer = KafkaProducer(bootstrap_servers = bootstrap_servers)
i=1
for i in range(100):
# Publish text in defined topic
    message_data = input("Enter message ")

    producer.send(topicName,str.encode(message_data)  )

# Print message
    print("Message Sent")
i=i+1

And this is my consumer code.

from kafka import KafkaConsumer
consumer = KafkaConsumer(bootstrap_servers=['localhost:9092'], auto_offset_reset='earliest')
consumer.subscribe(['First_Topic'])

for msg in consumer:
    print("Message from Producer on Topic - "+msg.topic+":"+ msg.value.decode())

Your consumer code looks fine. What is the issue? – OneCricketeer Dec 03 '20 at 19:17 — OneCricketeer, Dec 03 '20 at 19:17

score 0 · Answer 1 · answered May 01 '21 at 16:21

When producer produces a message on a Kafka topic it doesn't go to consumer directly instead it will go to Kafka broker first on a specific partition (which is like a persistent distributed queue).

How and how many consumers will get message will depend upon your consumer's setting. One or more consumers can subscribe to same topic/partition depending upon your subscription scheme. if you start multiple consumer using different consumer groupId (or no group Id at all) then all consumers will get same data. if same group id is used for all consumers only one consumer will get messages from one partition.

Implement Kafka Producer Consumer Chat with Queue

1 Answers1