1

Currently after I deploy Kafka to a machine, I need to run the following commands so it can be configured correctly:

docker run --net=azure-iot-edge --rm confluentinc/cp-kafka:5.1.2 kafka-topics --list --zookeeper zookeeper:2181
docker run --net=azure-iot-edge --rm confluentinc/cp-kafka:5.1.2 kafka-topics --create --topic new-project --zookeeper zookeeper:2181 --partitions 3 --replication-factor 1
docker run --net=azure-iot-edge --rm confluentinc/cp-kafka:5.1.2 kafka-topics --describe --zookeeper zookeeper:2181 --topic new-project

The problem is that i'm deploying this to a machine that does have Docker Run so I can no longer configure it.

In order to get around this, I'm trying to create a custom Kafka image that already have these configurations.

So I created a dockerfile that pull the image and run these configurations but that doesn't work because docker isn't available inside the dockerfile. I get the following error when I try to create the image: /bin/sh: 1: docker: not found.

This is how the dockerfile looks like: 

FROM confluentinc/cp-kafka:5.1.2

RUN docker run --net=azure-iot-edge --rm confluentinc/cp-kafka:5.1.2 kafka-topics --list --zookeeper zookeeper:2181
RUN docker run --net=azure-iot-edge --rm confluentinc/cp-kafka:5.1.2 kafka-topics --create --topic new-project --zookeeper zookeeper:2181 --partitions 3 --replication-factor 1
RUN docker run --net=azure-iot-edge --rm confluentinc/cp-kafka:5.1.2 kafka-topics --describe --zookeeper zookeeper:2181 --topic new-project

Anyone faced a similar issue and can offer some guidance?

Robin Moffatt
  • 30,382
  • 3
  • 65
  • 92
Rana
  • 1,675
  • 3
  • 25
  • 51
  • Get the tools from confluent.io or kafka.apache.org and run them directly. They don't specifically require Docker, especially the administrative commands you're looking at. – David Maze Oct 15 '19 at 23:35
  • (it's worth updating to 5.3.1, 5.1.2 was released quite a while ago) – Robin Moffatt Oct 16 '19 at 08:58
  • It would be useful if you edit your question with what you're trying to achieve, rather than _how_ you're trying to achieve it. `docker run` is just a way to invoke a tool inside the container. Where are you running docker? Are you just trying to pre-create a topic when you deploy your Kafka cluster? – Robin Moffatt Oct 16 '19 at 08:59
  • @RobinMoffatt Yes, I want the topic to exist when the container starts up. Is this possible? I'm trying to avoid Docker run because this require manual intervention and I'm trying to automate this. – Rana Oct 29 '19 at 18:44

1 Answers1

0

docker is not part of the base image confluentinc/cp-kafka:5.1.2 If you really need to use Docker insider Docker, suggest to start with this

However, from what you say:

after I deploy Kafka to a machine, I need to run the following commands so it can be configured correctly

I guess that you first want to run Docker container (deploy) of Kafka:

docker run --net=azure-iot-edge -d --name cp_kafka confluentinc/cp-kafka:5.1.2

and then execute the configuration commands (configure):

docker exec -i cp_kafka kafka-topics --list --zookeeper zookeeper:2181

...

rok
  • 9,403
  • 17
  • 70
  • 126