2

I am running buildbot which is a CI tool on an EC2 machine. It's currently running as docker containers one for buildbot master and one for buildbot worker. Inside buildbot worker, I have to again run docker for building images and running containers.

After doing some research on how to best do this, I have mounted the docker sock file from the host machine to the buildbot worker container. Now from inside the buildbot worker, I am able to connect to the host docker daemon and use the build cache.

Main problem now is that inside the buildbot worker, I have a docker compose file in which for one service, I am mounting a file like this ./configs/my.cnf:/etc/my.cnf but it is failing. And doing some more research, it's because the configs/my.cnf is relative to the buildbot worker directory and since I am using the host docker daemon which resolves the files using the host paths, it is not able to find the file.

I am not able to figure out on how to best do this. There were some suggestions on using the data volumes for this, but I am not sure on how best to use those. Any idea on how we can do this?

David Maze
  • 130,717
  • 29
  • 175
  • 215
Kishu Agarwal
  • 378
  • 3
  • 8

1 Answers1

1

Do you have any control over the creation of the buildbot worker? Can you control the buildbot worker directory.

export BUILD_BOT_DIR=$(mktemp -d) &&
    docker container create -v /var/run/docker.sock:/var/run/docker.sock -v ${BUILD_BOT_DIR}:${BUILD_BOT_DIR} -e BUILD_BOT_DIR ...

In this scenario, the path './configs/my:conf' points to the same file on both the container and the host.

emory
  • 10,725
  • 2
  • 30
  • 58
  • wouldn't specifying only `-v ${BUILD_BOT_DIR}` only create a volume and not a bind mount? I mean the files inside the buildbot worker would still be not resolved on the host daemon as it was a volume and not a bind mount with the same directory path. – Kishu Agarwal Jun 20 '18 at 17:35
  • and one more problem is that the buildbot worker directory already contains some files which I don't want to overwrite. Bind mount would overwrite the files – Kishu Agarwal Jun 20 '18 at 17:38
  • @KishuAgarwal I thought `-v ${BUILD_BOT_DIR}` was short hand for `-v ${BUILD_BOT_DIR}:${BUILD_BOT_DIR}`, but I think I was wrong about that. – emory Jun 23 '18 at 01:36
  • do you know of another way to achieve this? – Kishu Agarwal Jun 23 '18 at 13:53
  • The only way to achieve this is to make the files you will mount available to your docker host. So you will have to mount a volume from the docker host to the buildbot worker and then put the relevant files in the mounted volume. When you then start containers with `docker compose up`, in the volumes definitions, you have to point to the path on the **docker host**. – Michal Trojanowski Dec 13 '22 at 11:17