How set up test containers for paraller runwith docker compose?

Question

Here is docker-compose file:

  app:
    image: myimage
    depends_on:
      - nsqd
      - localstack
    command: ["run.sh"]
    environment:
      - "DYNAMODB=http://localstack:4569"
    ports:
      - 8080:8080

  nsqd:
    image: nsqio/nsq
    command: /run
    ports:
      - "4150:4150"
      - "4151:4151"

  localstack:
    image: localstack/localstack:latest
    ports:
      - 4569:4569
    environment:
      SERVICES: dynamodb
      DATA_DIR: /tmp/localstack/data
      HOSTNAME: localstack

This compose file is run in java junit test before any test method is run :

  @Before
  public void setUp() throws Exception {
        new DockerComposeContainer(new File("docker-compose.yaml"))
                .withExposedService("nsqd", 4150, Wait.forListeningPort())
                .withExposedService("localstack", 4569, Wait.forListeningPort())
                .withExposedService("app", 8080, Wait.forListeningPort())
                .start();
  }

When all test methods run one by one there are no problems at all. But when I try to run more than 2 test same time I got errors like that:

ERROR: for localstack  Cannot start service localstack: driver failed programming external connectivity on endpoint hwfdrbmwpwn1_localstack_1 (e33d2a3098e74b1b8d87e3e595d9d9504ccddd4fe9c0605b20ebd3f22f50daa5): Bind for 0.0.0.0:4569 failed: port is already allocated
ERROR: for nsqlookupd  Cannot start service nsqlookupd: driver failed programming external connectivity on endpoint hwfdrbmwpwn1_nsqlookupd_1 (fe62cec02a23a184d65b3f02776a14d77fdfbe639645ea0a11e07e8f11010e37): Bind for 0.0.0.0:4161 failed: port is already allocated

And those port differs from withExposedService function. From other side all service from compose file started in isolated network so there are should not be any conflicts but they exist. Can any bpody explain what is going on with ports? What is the additional config should be provided to testcontainers to run docker-compose services multiple times same time?

score 0 · Answer 1 · answered Aug 14 '19 at 11:15

0

The port defined with withExposedService is from the internal view of a container. Testcontainers will bind that port to a random external port. Have a read here:

https://www.testcontainers.org/features/networking

Do you also stop your docker compose containers before each test method?

I also would suggest to remove the port mapping from your docker compose file as it is not necessary with testcontainers:

Note that it is not necessary to define ports to be exposed in the YAML file; this would inhibit reuse/inclusion of the file in other contexts.

Taken from: https://www.testcontainers.org/modules/docker_compose/

answered Aug 14 '19 at 11:15

jwi

1,116
1
18
38

`Do you also stop your docker compose containers before each test method?` - this how it works and what I should avoid. Yes `start -> test -> stop` works. But I want to start multiple `docker-compose` with many random ports and invoke test methods in parallel not one-by-one and this is a problem. – Cherry Sep 05 '19 at 06:35
Mmh, I don't see the problem. Testcontainers will map your docker compose container on random ports on your host system, so a parallel run of multiple docker compose containers shouldn't be a problem. – jwi Sep 06 '19 at 07:21
`Mmh, I don't see the problem` have you tried the code? – Cherry Sep 10 '19 at 08:49

score 0 · Answer 2 · answered Apr 03 '20 at 17:23

If I understood your setup correctly you want start and stop your docker-compose containers for each test and do so with potentially several different docker-compose-files in different tests (or different tests with the same file) at the same time.

There exists an alternative library, Docker-Compose-Rule of Palantir!.

There is actually a collaboration going on between the two (testContainers and Palantir) since testContainers is far more generic, but Palantir library was more in depth using docker-compose. The Collaboration started in 2018 but for now the library is still maintained so it might still has a specialization advantage which solves your problem.

How set up test containers for paraller runwith docker compose?

2 Answers2