How to avoid installing jq/docker into container running inside of WITH DOCKER block?

Question

I build JVM-based project and would like to have an Earthly target with integration steps similar to this one using WITH DOCKER ... END syntax. WITH DOCKER command is needed to have a real database instance available in the context of integration tests execution.

Since I work on a JVM project, my base image for executing any commands related to build system is: FROM bellsoft/liberica-openjdk-alpine:17. The thing which I find suboptimal is that for any command running inside of WITH DOCKER ... END block Earthly check for presence of jq and docker/docker-compose projects. Each time I execute integration tests on CI node, jq and docker get installed, while they are completely useless in my WITH DOCKER usage scenario.

Is there a way to disable their installation? Right now, as a workaround, I consider adding jq and docker to my base bellsoft/liberica-openjdk-alpine:17 builder-image to make docker/jq installation *cached*

score 1 · Answer 1 · answered Mar 27 '23 at 18:54

You can use the INSTALL_DIND UDC for this. The best practices guide touches on this under the # Better example.

This allows you to use a custom base image and also install dependencies on top, and then do any cache-busting operations (like COPY) after.

FROM some-other-image:latest
DO github.com/earthly/lib+INSTALL_DIND
COPY ...
WITH DOCKER ...
   RUN ...
END

Another good option is to use the earthly/dind image and instead put the JDK-related things in another container that you then --load and run via docker run within the WITH DOCKER. However it sounds like this might not be a viable option for you.

Thank you! I would still prefer to have an option to skip installing these dependencies (skipping this would make my jvm-"builder" image smaller, which in turn would make cloud CIs to download less on each build). Yeah, I will go the first option and install docker/jq into my jvm-builder, because it's easier than installing JDK distribution I need in top of `earthly/dind` — Kirill, Mar 28 '23 at 03:50

score 0 · Accepted Answer · answered Mar 30 '23 at 05:25

Alex Couture-Beil recently provided good answer to this question on Earthly Slack

I think that's the best way to avoid installing dependencies on each build, just use the recommended earthly/dind:alpine for WITH DOCKER commands, it will only be downloaded once and run tests with docker run instead of RUN. Here is how I changed my pipeline now:

test:
    FROM earthly/dind:alpine # <-- recommended image for 'WITH DOCKER' commands, docker and jq are already there
    COPY docker-compose.yaml ./
    WITH DOCKER --compose docker-compose.yaml --service=mongo --load build-image=+compile
        RUN docker run --name tests-container --network=host build-image \
        ./gradlew test \
        && \
        docker cp tests-container:/build/ build/ # <-- get test execution results
    END
    SAVE ARTIFACT build/test-results

How to avoid installing jq/docker into container running inside of WITH DOCKER block?

2 Answers2