0

On my VPS I have 3 docker containers: zookeeper, kafka and my own java/spring boot service. My questions are:

  1. How should I configure kafka in compose file to be able to produce/consume messages both from the java service running as a docker container on the same VPS and from my local laptop?
  2. What kafka URL should I use in java service deployed to VPS (I tried 'localhost:29092' and 'kafka:29092' but with no luck - host not resorvable error)?
  3. What kafka URL should I use if I want to access it from my local laptop (I tried 'MY_VPS_IP_ADDRESS:29092' but I'm getting: 'LEADER_NOT_AVAILABLE')?

My docker compose file:

version: '3'

services:
        zookeeper:
                image: confluentinc/cp-zookeeper
                container_name: zookeeper
                ports:
                        - "2181:2181"
                environment:
                        ZOOKEEPER_CLIENT_PORT: 2181
        kafka:
                image: confluentinc/cp-kafka
                container_name: kafka
                ports:
                        - "29092:29092"
                environment:
                        KAFKA_LISTENERS: INSIDE://0.0.0.0:9092,OUTSIDE://0.0.0.0:29092
                        KAFKA_ADVERTISED_LISTENERS: INSIDE://kafka:9092,OUTSIDE://MY_VPS_IP_ADDRESS:29092
                        KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT
                        KAFKA_INTER_BROKER_LISTENER_NAME: INSIDE
                        KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
                depends_on:
                        - zookeeper

I did also try wurstmeister version of these images but with the same result.

Marek
  • 11
  • 1
  • As far as Compose is concerned, `OUTSIDE://MY_VPS_IP_ADDRESS:29092` + `29092:29092` host mapping is correct... You need to make sure your VPS allows external access on that port, then use that advertised IP address from any client. – OneCricketeer Mar 08 '22 at 22:10
  • If you want it to work within the Docker network, on the VPS, **and** as a local Compose service, then you need three listeners, or export `KAFKA_ADVERTISED_LISTENERS` to an `.env` file that is different depending on where you run `docker-compose` – OneCricketeer Mar 08 '22 at 22:11
  • For "host not resolvable" 1) `kafka:9092` is the correct "inside Docker network" address you have set, not `:29092`. 2) Your Java service needs to be in the same compose file to be part of the same bridge network between Kafka and Zookeeper if you don't add `networks` config – OneCricketeer Mar 08 '22 at 22:13

0 Answers0