Suppress docker networking with docker compose for test execution isolation

Question

How do I suppress docker-compose up mapping to the host IP range (for multiple instances)?

Context: I'm looking to run unit tests in Jenkins where we have a docker engine on the master and each slave.

I want to spin up some containers for a unit-integration test and shut them down after.

I want multiple tests to be able to run the same docker compose file simultaneously, without contention.

docker-compose -f "my-test-docker-compose.yml" -p "ab2de2f38c" up -d -t 100

and for the second execution:

docker-compose -f "my-test-docker-compose.yml" -p "34cd832b7" up -d -t 100

I find the first network binds to 0.0.0.0/< port > which causes conflicts when the second execution happens.

Cannot start service zookeeper: driver failed programming external connectivity on endpoint 34cd832b7_zookeeper_1 (c2b928530cfdc7aee17): Bind for 0.0.0.0:22181 failed: port is already allocated

Have tried adding networking to each service and at the bottom and then created the network with docker network create. No dice, thus far.

Ideally want a solution that doesn't require the docker-compose.yml file to change as this gives testers a burden.

Is there a way to suppress any mapping between the containers and the host's IP range and then get the IP from outside the network using docker ps and docker inspect in conjunction.

The ideal would be:

c, err := docker.New("my-test-docker-compose.yml")
//handle err
c.Up()
//connect to a service..
s, err := service1.New(c.GetSocket("serv:1234"))
//handle connection error

//do testing 

c.Down()

Any suggestions would be ideal. It may be the only options are to re-work every docker-compose.yml to remove all external network mappings, which is painful or to spawn a docker within a docker to completely isolate the network. Painful.

Any suggestions how to pretty much specify --network=none so we get no 0.0.0.0/ bindings would be great. I note there is a golang docker library (libcompose) in this area, but sadly it's no longer maintained.

Thank you,

alex

Answer 1

Ok I've solved this (credit is actually to a team mate).

The solution is to replace ports with expose in the docker-compose.yml file.

Sounds simple but this has the following result:

Before: (HTML table)

 <table border=1>
<tr><th>CONTAINER_ID</th><th>....</th><th>PORTS</th></tr>
<tr><td>75a45ac5ef62</td><td>....</td><td>0.0.0.0:9094->9094/tcp, 0.0.0.0:33919->9092/tcp, 0.0.0.0:33918->9093/tcp, 0.0.0.0:33917->22888/tcp</td></tr>
<tr><td>1920786d77ed</td><td>....</td><td>0.0.0.0:9092->9092/tcp, 0.0.0.0:33909->9093/tcp, 0.0.0.0:33908->9094/tcp</td></tr>
</table>

After: (HTML table)

    <table border=1>
    <tr><th>CONTAINER_ID</th><th>....</th><th>PORTS</th></tr>
    <tr><td>75a45ac5ef62</td><td>....</td><td>9092-9094/tcp, 22888/tcp, 32888/tcp, 43888/tcp</td></tr>
    <tr><td>1920786d77ed</td><td>....</td><td>2181/tcp, 2888/tcp, 3888/tcp, 22888/tcp, 33888/tcp, 42181/tcp, 43888/tcp</td></tr>
    </table>

So: "0.0.0.0:9094->9094/tcp" becomes "9094-9094/tcp".

The issue was that 'ports' pushed every registered port to the host machine on IP 0.0.0.0. Expose doesn't do this. This way each container in the network can still see each other and resolve but this isn't published out to the host.

This can now be used with docker ps => docker inspect to get the relevant IP for each host declared in the docker-compose.yml file and allow code to run each test in isolation including tests all calling the same file.

Result.