cannot copy from one stage to another in multi stage docker build

Question

I want to create and execute performance tests against the application. For now, my idea was to use multi-stage build on the first stage to build the application, on the second - build the performance testing and start both application and performance tests in the same container.

My Dockerfile looks like this

# build stage
FROM gradle:jdk11
ARG ARTIFACT_PATH=json-comparison-application/build/libs
ARG ARTIFACT_NEW=app.jar
ARG ARTIFACT_OLD=json-comparison-application-0.0.1-SNAPSHOT.jar
RUN echo ${ARTIFACT_PATH}
RUN apt-get install git && git clone https://github.com/xxxx/json-comparison.git
WORKDIR json-comparison
RUN chmod +x gradlew && \
    ./gradlew clean build -x pmdMain -x spotbugsMain -x checkstyleMain --no-daemon && \
    cd ${ARTIFACT_PATH} && mv ${ARTIFACT_OLD} ${ARTIFACT_NEW}

# performance test stage
FROM ubuntu:18.04
# simplified adoptopenjdk/11 without CMD entry point, probably better move to some external dockerfile
ARG ESUM='6dd0c9c8a740e6c19149e98034fba8e368fd9aa16ab417aa636854d40db1a161'
ARG BINARY_URL='https://github.com/AdoptOpenJDK/openjdk11-binaries/releases/download/jdk-11.0.5%2B10/OpenJDK11U-jdk_x64_linux_hotspot_11.0.5_10.tar.gz'
ENV LANG='en_US.UTF-8' LANGUAGE='en_US:en' LC_ALL='en_US.UTF-8'
ENV JAVA_VERSION jdk-11.0.5+10
RUN apt-get update \
    && apt-get install -y --no-install-recommends curl ca-certificates fontconfig locales \
    && echo "en_US.UTF-8 UTF-8" >> /etc/locale.gen \
    && locale-gen en_US.UTF-8 \
    && rm -rf /var/lib/apt/lists/*
RUN set -eux; \
    curl -LfsSo /tmp/openjdk.tar.gz ${BINARY_URL}; \
    echo "${ESUM} */tmp/openjdk.tar.gz" | sha256sum -c -; \
    mkdir -p /opt/java/openjdk; \
    cd /opt/java/openjdk; \
    tar -xf /tmp/openjdk.tar.gz --strip-components=1; \
    rm -rf /tmp/openjdk.tar.gz;
ENV JAVA_HOME=/opt/java/openjdk \
    PATH="/opt/java/openjdk/bin:$PATH"
# custom part of the stage
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get install -y python default-jre-headless python-tk python-pip python-dev \
                       libxml2-dev libxslt-dev zlib1g-dev net-tools && \
    pip install bzt
WORKDIR /home/tmp
ARG ARTIFACT_PATH=json-comparison/json-comparison-application/build/libs
ARG ARTIFACT_NEW=app.jar
RUN echo ${ARTIFACT_PATH}
COPY --from=0 /${ARTIFACT_PATH}/${ARTIFACT_NEW} .
# prototype for test
CMD ["bzt", "quick_test.yml"]

But it fails during build with error

Step 23/24 : COPY --from=0 /${ARTIFACT_PATH}/${ARTIFACT_NEW} .
COPY failed: stat /var/lib/docker/overlay2/f227e7b77fba105ba0769aa355458900d202add59c98583f0fd0936cbf4dfc11/merged/json-comparison/json-comparison-application/build/libs/app.jar: no such file or directory

What is the problem?

ErikMD · Accepted Answer · 2019-12-27T23:10:46.783

At first sight the problem comes from the (absolute vs. relative) paths of the .jar in your two stages.

Can you verify that they are the same?

For example, you may replace the last command of the first stage with

RUN […] && \
  cd ${ARTIFACT_PATH} && mv ${ARTIFACT_OLD} ${ARTIFACT_NEW} && readlink -f ${ARTIFACT_NEW}

and relaunch the build. If you don't obtain

/json-comparison/json-comparison-application/build/libs/app.jar

but a longer path, then you'll know the correct path to put in ARTIFACT_PATH within the second stage…

Alternatively, you could just get rid of the relative path in the first stage and replace WORKDIR json-comparison with:

WORKDIR /json-comparison

As an aside, it can be useful to name your build stages in the following way:

FROM gradle:jdk11 as builder
[…]

FROM ubuntu:18.04 as test
[…]

Then, this allows you to only build the first stage by running:

$ docker build --target builder -t $IMAGE_NAME .

It seems the artifact is on `/home/gradle/json-comparison...` path probably because `gradle` image sets `WORKDIR` as `/home/gradle` — lapots, Dec 28 '19 at 20:27
@lapots Yes, actually `WORKDIR $dir` can be viewed as a Dockerfile equivalent to `mkdir -p $dir && cd $dir`. And I'd say a good practice is to prefer using an absolute path for $dir instead of a relative path. — ErikMD, Dec 28 '19 at 20:37

cannot copy from one stage to another in multi stage docker build

1 Answers1