3

I need to connect to Kafka broker in python using SASL/SCRAM-SHA-256 authentication. I use the kafka-python (2.0.1) python library which allows this kind of connection/authentication.

This is an example of code, but i get the error ... not Brokers available 

prod = KafkaProducer(bootstrap_servers='server:9093',
                     security_protocol='SASL_SSL',
                     ssl_certfile='path to ca in pem format',
                     sasl_mechanism='SCRAM-SHA-256',
                     sasl_plain_username='xxx',
                     sasl_plain_password='xxx')

Getting the following error

    ssl.SSLError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed (_ssl.c:897)
20-04-06 09:58:52 - root - 1 - 140658864072256 - MainThread - ERROR - Error in connection to kafka
Traceback (most recent call last):
  File "/*****/*****/dao/kafka_dao.py", line 44, in __init__
    **self.consumer_additional_params)
  File "/opt/app-root/lib/python3.6/site-packages/kafka/consumer/group.py", line 355, in __init__
    self._client = KafkaClient(metrics=self._metrics, **self.config)
  File "/opt/app-root/lib/python3.6/site-packages/kafka/client_async.py", line 242, in __init__
    self.config['api_version'] = self.check_version(timeout=check_timeout)
  File "/opt/app-root/lib/python3.6/site-packages/kafka/client_async.py", line 907, in check_version
    version = conn.check_version(timeout=remaining, strict=strict, topics=list(self.config['bootstrap_topics_filter']))
  File "/opt/app-root/lib/python3.6/site-packages/kafka/conn.py", line 1228, in check_version
    if not self.connect_blocking(timeout_at - time.time()):
  File "/opt/app-root/lib/python3.6/site-packages/kafka/conn.py", line 337, in connect_blocking
    self.connect()
  File "/opt/app-root/lib/python3.6/site-packages/kafka/conn.py", line 426, in connect
    if self._try_handshake():
  File "/opt/app-root/lib/python3.6/site-packages/kafka/conn.py", line 505, in _try_handshake
    self._sock.do_handshake()
  File "/usr/lib64/python3.6/ssl.py", line 1033, in do_handshake
    self._sslobj.do_handshake()
  File "/usr/lib64/python3.6/ssl.py", line 645, in do_handshake
    self._sslobj.do_handshake()

Where am I doing wrong??

Alfonso
  • 113
  • 9
  • Unless you want a million wild guesses, please provide producer and broker logs. – mazaneicha Apr 04 '20 at 13:40
  • ok logs added!!!! – Alfonso Apr 06 '20 at 10:06
  • So "_certificate verify failed_" basically says your client is unable to validate server certificate while trying to establish SSL connection. You will probably need to update client's `truststore`. Other option is to disable certificate validation, but it is not a secure solution. I think there are numerous SO posts around on these subjects. – mazaneicha Apr 06 '20 at 12:33
  • what do u mean for "update client's truststore" ? anyway, what's the difference between ssl_certfile and ssl_cafile fields? from the documentation they seem to be quite similar... I saw in most of StackOverflow thread that people use both fields. They generate the CAroot.pem and certificate.pem from jks...what if i have already the certificate.pem? – Alfonso Apr 06 '20 at 14:15

0 Answers0