Kafka with Docker Problems

Question

I am trying to deploy kafka by docker-compose file or evenby installing kafka image and run it manually. Both of these steps are giving to me that error when I start to run kafka server(broker)

INFO Initiating client connection, connectString=188.226.151.167:2181 sessionTimeout=6000 watcher=org.I0Itec.zkclient.ZkClient@323b36e0 (org.apache.zookeeper.ZooKeeper) [2017-05-16 13:44:49,903] INFO Waiting for keeper state SyncConnected (org.I0Itec.zkclient.ZkClient) [2017-05-16 13:44:49,909] INFO Opening socket connection to server 188.226.151.167/188.226.151.167:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn) [2017-05-16 13:44:55,904] INFO Terminate ZkClient event thread. (org.I0Itec.zkclient.ZkEventThread)

Could anyone has clear explanation of what happens and the clear explanation about how to fix it

Can you please share your docker-compose?Also, please share the docker version and OS you are working on. — humblebee, May 16 '17 at 17:53
docker compose file version: '2' services: zookeeper: image: wurstmeister/zookeeper ports: - "2181:2181" kafka: build: . ports: - "9092:9092" environment: KAFKA_ADVERTISED_HOST_NAME: 192.168.99.100 KAFKA_CREATE_TOPICS: "test:1:1" KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181 volumes: - /var/run/docker.sock:/var/run/docker.sock — Eman, May 17 '17 at 09:50
OS version : Ubuntu 16.04 Docker Version: Docker version 17.04.0-ce — Eman, May 17 '17 at 09:53

score 0 · Answer 1 · answered May 17 '17 at 10:01

Try this:

kafka:
  network_mode: 'bridge'
  image: wurstmeister/kafka
  links:
    - zookeeper
  environment:
    HOSTNAME_COMMAND: "hostname -I | awk -F' ' '{print $$1}'"
    KAFKA_ADVERTISED_PORT: 9092
    KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
zookeeper:
  network_mode: 'bridge'
  image: wurstmeister/zookeeper

score 0 · Answer 2 · answered May 17 '17 at 12:41

0

I have setup my docker compose as this and everything works perfectly:

version: '2'
services:
  zookeeper:
    image: wurstmeister/zookeeper
    ports:
      - 2181:2181
    hostname: zookeeper
    container_name: zookeeper
    volumes:
      - /tmp/docker/zk1/logs:/logs
      - /tmp/docker/zk1/data:/data
  kafka:
    image: wurstmeister/kafka
    ports:
      - 9092:9092
    depends_on:
      - zookeeper
    environment:
      KAFKA_CREATE_TOPICS: "test1:1:1,test2:1:1"
      HOSTNAME_COMMAND: "route -n | awk '/UG[ \t]/{print $$2}'"
      KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
    volumes:
      - /var/run/docker.sock:/var/run/docker.sock
      - /tmp/docker/kafka/logs:/logs
      - /tmp/docker/kafka/data:/data
      - ./tmp/docker/kafka/kafka-logs:/kafka
    hostname: kafka
    container_name: kafka

I have setup the ports to be like '2181:2181' and not just '2181' so that to be accessible outside my container.

I have also set the volumes so that every time I restart the container to remember all the data from previous time.

answered May 17 '17 at 12:41

Hakuna Matata

119
3
14

When I made this docker compose file up these errors appears to me Error:KeeperErrorCode = NoNode for /brokers/topics/test1/partitions/0 and this error caught end of stream exception zookeeper1 | EndOfStreamException: Unable to read additional data from client sessionid 0x15c1a74ce3c0002, likely client has closed socket – Eman May 18 '17 at 07:38
Are you sure this docker compose file doesn't depend on any another file ? – Eman May 18 '17 at 07:43
I want know what is the [link] HOSTNAME_COMMAND ? Because when I tried to type this command [command] route -n | awk '/UG[ \t]/{print $$2}' in terminal nothing appears to me Is this the local host IP or docker machine IP ? – Eman May 18 '17 at 07:57
I don't know why you get this errors. I just created a test directory with a docker-compose.yml with the code that I have posted above and it perfectly works. You do not necessarily need to set the KAFKA_CREATE_TOPICS though, you can do that through your application initialization. To be honest I was testing that command but I use other topics that I dynamically create through my app. As for the command I found it on the [wurstmeister/kafka](https://hub.docker.com/r/wurstmeister/kafka/) and without this HOSTNAME_COMMAND it was not working so I set it up as suggested. – Hakuna Matata May 18 '17 at 09:24
SOLVED By disabling docker machine firewall command : ufw disable – Eman May 18 '17 at 11:05

score 0 · Answer 3 · answered May 18 '17 at 11:08

0

It solved all kafka with docker connection by outside machines by disabling firewall connection Command: ufw disable

answered May 18 '17 at 11:08

Eman

111
1
4
14

Kafka with Docker Problems

3 Answers3

Linked