1

I have dockerized my project that using kafka for communication, I am using python on consume part of my project, I used two different kafka image because of the NoBrokersAvaible error and I tried all of the combinations I guess. I want to consume messages from kafka but my consumer cannot connect with my kafka brokers.

Console Logs

Traceback (most recent call last):
 File "main.py", line 15, in <module>
  kafka_consumer = kafka_client.prepare_kafka_consumer()
 File "/-----/client/kafka_client.py", line 17, in    prepare_kafka_consumer
  consumer = KafkaConsumer(self.topic,
 File "/usr/local/lib/python3.8/site-packages/kafka/consumer/group.py", line 356, in __init__
  self._client = KafkaClient(metrics=self._metrics, **self.config)
 File "/usr/local/lib/python3.8/site-packages/kafka/client_async.py", line 244, in __init__
  self.config['api_version'] = self.check_version(timeout=check_timeout)
 File "/usr/local/lib/python3.8/site-packages/kafka/client_async.py", line 900, in check_version
  raise Errors.NoBrokersAvailable()
 kafka.errors.NoBrokersAvailable: NoBrokersAvailable

Docker Compose

services:
  zookeeper:
    image: confluentinc/cp-zookeeper:3.2.0
    container_name: zookeeper
    hostname: zookeeper
    ports:
      - "2181:2181"
    environment:
      ZOOKEEPER_CLIENT_PORT: 2181
      ZOOKEEPER_TICK_TIME: 2000
    restart: always

  kafka:
    image: confluentinc/cp-kafka:3.2.0
    hostname: kafka
    container_name: kafka
    depends_on:
      - zookeeper
    ports:
      - '9092:9092'
    environment:
      KAFKA_ADVERTISED_LISTENERS: LISTENER_DOCKER_INTERNAL://kafka:19091,LISTENER_DOCKER_EXTERNAL://${DOCKER_HOST_IP:-127.0.0.1}:9093
      KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: LISTENER_DOCKER_INTERNAL:PLAINTEXT,LISTENER_DOCKER_EXTERNAL:PLAINTEXT
      KAFKA_INTER_BROKER_LISTENER_NAME: LISTENER_DOCKER_INTERNAL
      KAFKA_ZOOKEEPER_CONNECT: "zookeeper:2181"
      KAFKA_BROKER_ID: 1
      KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
    restart: always

  app:
    build:
        context: .
        dockerfile: DockerFile
    ports:
      - "8080:8080"
    depends_on:
      - kafka

Config.py

class Settings:
    KAFKA_BROKERS = ['kafka:19091']
    KAFKA_TOPIC = "topic"
    MAIL_USERNAME = "mail"
    MAIL_PASSWORD = "pass"


settings = Settings()

I am not able to use my app and kafka in same container properly.

  • Your exposed ports do not match the port from configuration. That's not necessarily required on a non-default bridge, but something to check. – Klaus D. Aug 10 '22 at 08:29

0 Answers0