set docker host ip (advertised.host.name) for mac and ubuntu in docker-compose.yml

Question

I have a web application running in docker container, and a kafka broker running in another docker container. I am using docker-compose to setup up the docker containers. but I found that it is a little difficult to config the networking for docker containers. so, I posted some questions: Java & Kafka: Connection to node 999 could not be established. Broker may not be available

I got some hits to setup advertised.host.name the from here Cannot produce message to kafka from service running in docker. Now, my web application can produce the messages to the kafka docker container. But I got another issue: I can get docker host ip by ifconfig docker0, and hardcoded it in docker-compose.yml. I have no idea how to get it in mac. Our docker-compose files are for dev, and they should be run in lots of developers' computers. so, we need some reliable way to get the ip automatically and dynamically by some programs.

Any ideas? thanks

Why can't you just `-p 9092:9092` on the Mac? The only reliable way to get a port mapped to an external network (on a Mac) is to use Docker Machine — OneCricketeer, Jan 28 '18 at 15:52
@cricket_007, I do not know how to set host ip in docker-compose.yml. port 9092 is working fine. — BAE, Jan 28 '18 at 15:54
Also, not clear why you're setting that property https://stackoverflow.com/a/37844327/2308683 — OneCricketeer, Jan 28 '18 at 15:55
Docker Compose can accept environmental variables... `${HOST_IP}`, but again, not sure why you need it. You can have the container listening on `0.0.0.0`, then you forward the Mac port to the container — OneCricketeer, Jan 28 '18 at 15:56
You might be interested in this example https://github.com/confluentinc/cp-docker-images/blob/4.0.x/examples/cp-all-in-one/docker-compose.yml — OneCricketeer, Jan 28 '18 at 16:01

score 0 · Answer 1 · answered Jan 23 '22 at 15:24

For clients connecting to any Kafka node, Kafka will answer back which hostname to connect too. If you run a Kafka node a docker-compose environment and connect from 'outside' docker - which is typically when doing development that hostname might be incorrect.

If that's the problem, you could solve it by using the following environment for kafka:

kafka:
  image: <my kafka image>
  ports:
  - 9094:9094
  links:
  - zookeeper
  environment:
    - "KAFKA_ADVERTISED_HOST_NAME=${KAFKA_ADVERTISED_HOST_NAME}"
    - "ZOOKEEPER_CONNECT=zookeeper:2181"
    - "KAFKA_ADVERTISED_LISTENERS=INSIDE://:9092,OUTSIDE://${KAFKA_ADVERTISED_HOST_NAME}:9094"
    - "KAFKA_LISTENERS=INSIDE://:9092,OUTSIDE://:9094"
    - "KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=INSIDE:PLAINTEXT,OUTSIDE:PLAINTEXT"
    - "KAFKA_INTER_BROKER_LISTENER_NAME=INSIDE"

and also set environment variable KAFKA_ADVERTISED_HOST_NAME to the hostname of your laptop.

Then you can connect to the kafka that's running your docker environment by using the address localhost:9094 Any application that's running inside the same docker environment can connect using the internal networking to kafka:9092

On Windows and MacOs you might have some internal environment variable like HOST_IP or HOSTNAME that's already configured, but to avoid hassle I'd configure KAFKA_ADVERTISED_HOST_NAME in my default environment anyway to my laptop's hostname.

Would this be effectively the same as specifying a `hostname: ${KAFKA_ADVERTISED_HOST_NAME}` field in the docker compose file above? Struggling to find the difference between the `hostname` parameter and environment variable used here. — decoder247, Jan 25 '22 at 11:46
Afaik that wouldn't be the same. KAFKA_ADVERTISED_HOST_NAME is a setting for kafka, which hostname the listeners return when an client connect. — Gerbrand, Jan 26 '22 at 12:32

set docker host ip (advertised.host.name) for mac and ubuntu in docker-compose.yml

1 Answers1