I am using the KafkaProducer libraray to publish messages such as follows:
from kafka import KafkaProducer
ssl_produce = KafkaProducer(bootstrap_servers='xxxx:xxx',
security_protocol='SASL_SSL',
sasl_mechanism='SCRAM-SHA-512',
sasl_plain_username='xxxx',
sasl_plain_password='xxxxx')
ssl_produce.send('xxx', key=b'xxx', value=message_data)
This approach works fine. Now, I try to switch to the confluent Kafka lib such as follows:
from confluent_kafka import Producer, KafkaException
from confluent_kafka.admin import AdminClient, NewTopic
from confluent_kafka.serialization import StringSerializer
# Kafka broker properties
bootstrap_servers = 'xxx:xxx'
security_protocol = 'sasl_ssl'
sasl_mechanism = 'SCRAM-SHA-512'
sasl_username = 'xxxx'
sasl_password = 'xxx'
# Kafka topic
topic_name = 'xxxx'
# Create the Kafka producer configuration
producer_config = {
'bootstrap.servers': bootstrap_servers,
'security.protocol': security_protocol,
'sasl.mechanism': sasl_mechanism,
'sasl.username': sasl_username,
'sasl.password': sasl_password,
'auto.offset.reset': 'earliest', # Read messages from the beginning of the topic
'default.topic.config': {
'auto.offset.reset': 'earliest'
}
}
# Create the Kafka producer
producer = Producer(producer_config)
# Publish a message to Kafka
def publish_message(producer, topic, message):
try:
producer.produce(topic=topic, value=message)
producer.flush()
print('Message published successfully.')
except KafkaException as e:
print('Failed to publish message:', str(e))
admin_client = AdminClient(producer_config)
# Publish a sample message
message = 'Hello, Kafka!'
publish_message(producer, topic_name, message)
This code hangs and the message is not published. I am using the same topic and the same username/password. What I am doing wrong?