Docker in docker - how to ensure it's cleaned up after itself?

Question

I've built a Swarm-based Docker setup for our on-prem DevOps pipelines. Docker-based pipeline agents are started and can perform build operations. I also have a few of those agents capable of building new docker images - this was enabled by binding \\.\pipe\docker_engine from the host with these containers.

This generally works... however in case there's an issue with the build process it'll likely leave a lot of garbage behind. This can partially be alleviated by using --force-rm. But ideally I'd like to have the containers clean up after themselves so that the next run is "pristine" regardless of what was ran inside it. I'd also like to let these special containers launch new containers for more complex CI pipelines, but again - I'm worried about them not cleaning up after themselves. Note: I'm less worried about security since this is all "in-house" stuff.

Is it possible to have a container which can launch nested containers inside of itself whilst making sure that if this top container is stopped & removed then all of the stuff created by this container will be stopped & removed as well?

score 0 · Answer 1 · answered Apr 21 '23 at 17:35

0

If you run the containers in a dind (Docker in Docker) container as background, you can store the process id and ensure proper cleanup. The cleanup can be hooked onto the interrupt or termination signals.

Here is an example script which you would have to execute on dind-container startup:

#!/bin/sh
cnt_name="dindContainer"

# intercept SIGINT and SIGTERM and hook a function onto them
trap 'cleanup' INT TERM

cleanup() {
    if [ -n "$ctr_proc" ]; then
        kill -TERM "$ctr_proc"
    else
        docker stop "$cnt_name" > /dev/null 2>&1
        sleep 5
    fi
}

cleanup
docker run \
    --rm \
    --name="$cnt_name" \
    alpine \
        sleep 1000 &

# store the container process id and wait for it to finish
ctr_proc=$!
wait "$ctr_proc"

answered Apr 21 '23 at 17:35

Synertry

83
1
3

I can try and use this for our Linux-based DIND dockers... but the question was primarily focused on Windows DIND (see tags). – MBender Apr 25 '23 at 11:08
@MBender I see. Signal event handling is unfortunately messy with Windows containers. A Windows container treats a SIGTERM like CTRL_SHUTDOWN_EVENT (SIGKILL), see [here](https://github.com/moby/moby/issues/25982#issuecomment-806001635). You would have to write you on shutdown handler in most likely C#. In either way, as it seems you also want to cleanup artefacts in your swarm like created images, you would be better off with a global prune cmd like [here](https://github.com/moby/moby/issues/31254#issuecomment-464668235). Adapt it to Windows of course. – Synertry Apr 25 '23 at 23:37

Docker in docker - how to ensure it's cleaned up after itself?

1 Answers1