127

I'm writing a multi-stage Dockerfile for the darshan utils:

ARG DARSHAN_VER=3.1.6

FROM fedora:29 as build
RUN dnf install -y \
        gcc \
        make \
        bzip2 bzip2-devel zlib zlib-devel
RUN curl -O "ftp://ftp.mcs.anl.gov/pub/darshan/releases/darshan-${DARSHAN_VER}.tar.gz" \
    && tar ...


FROM fedora:29
COPY --from=build "/usr/local/darshan-${DARSHAN_VER}" "/usr/local/darshan-${DARSHAN_VER}"
...

I build it with docker build -t darshan-util:3.6.1 . and the error I get is:

Step 5/10 : RUN curl -O "ftp://ftp.mcs.anl.gov/pub/darshan/releases/darshan-${DARSHAN_VER}.tar.gz"     && tar ...

 ---> Running in 9943cce1669c
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
...
curl: (78) RETR response: 550
The command '/bin/sh -c curl -O "ftp://ftp.mcs.anl.gov/pub/darshan/releases/darshan-${DARSHAN_VER}.tar.gz"     && tar ...' returned a non-zero code: 78

I'd like to reuse the same ARG in both stages, so that I can define a default build variable just once. If I duplicate ARG in both stages, just below the two FROMs, it builds correctly.

What is the correct way to define a "global" multi-stage ARG variable with a default?

Alberto Chiusole
  • 2,204
  • 2
  • 16
  • 26
  • If the first stage just downloads and unpacks the file, why does it need all those dev tools? Also, could you remove the version from the name so subsequent steps don't need to know it? – jonrsharpe Dec 08 '18 at 10:52
  • @jonrsharpe after untarring the archive there's a configure + make + make install, but I avoided to report it to keep the example minimal. – Alberto Chiusole Dec 08 '18 at 18:32

2 Answers2

227

ARGs only last for the build phase of a single image. For the multistage, renew the ARG by simply stating:

ARG DARSHAN_VER

after your FROM instructions.

cf. https://docs.docker.com/engine/reference/builder/#arg

ARG DARSHAN_VER=3.1.6

FROM fedora:29 as build
ARG DARSHAN_VER
RUN dnf install -y \
        gcc \
        make \
        bzip2 bzip2-devel zlib zlib-devel
RUN curl -O "ftp://ftp.mcs.anl.gov/pub/darshan/releases/darshan-${DARSHAN_VER}.tar.gz" \
    && tar ...


FROM fedora:29
ARG DARSHAN_VER
COPY --from=build "/usr/local/darshan-${DARSHAN_VER}" "/usr/local/darshan-${DARSHAN_VER}"
...

You will notice how I declared the initial value at the top of the script, and pull it in on each image.

Richard Barber
  • 5,257
  • 2
  • 15
  • 26
  • 15
    Perfect! I didn't notice the paragraph `An ARG instruction goes out of scope at the end of the build stage where it was defined. To use an arg in multiple stages, each stage must include the ARG instruction.` even though I read that doc twice. – Alberto Chiusole Dec 08 '18 at 19:37
  • 1
    Doesn't even remotely work. Still empty. – user64204 Oct 06 '22 at 09:05
  • @user64204 what is not working and what is still empty? – Richard Barber Oct 07 '22 at 17:13
  • 3
    One key to getting this to work the way I wanted: the first ARG needs to be declared at the top of the file, outside the scope of any stage and before the first FROM. – John Dibling Dec 31 '22 at 23:30
  • 2
    @JohnDibling, yes, thank you for clarifying that, this should be added to the current answer – SweetFeet Feb 21 '23 at 20:46
  • Indeed the structure shown can be summarized as a value declaration at the top of the script, with the value being pulled into each image. – Richard Barber Jul 12 '23 at 01:07
35

Here are quotes from the documentation:

An ARG instruction goes out of scope at the end of the build stage where it was defined. To use an arg in multiple stages, each stage must include the ARG instruction.

https://docs.docker.com/engine/reference/builder/#scope

An ARG declared before a FROM is outside of a build stage, so it can’t be used in any instruction after a FROM. To use the default value of an ARG declared before the first FROM use an ARG instruction without a value inside of a build stage

https://docs.docker.com/engine/reference/builder/#understand-how-arg-and-from-interact

Felix K.
  • 14,171
  • 9
  • 58
  • 72
  • 3
    "global" multi-stage ARG variables with a default do exist (even at the time of your answer), see e.g. here: https://github.com/moby/moby/issues/37622#issuecomment-412101935 – Axel Mar 16 '21 at 13:27
  • 1
    Hi Axel, thanks for notifying me about this, you are right. I removed my original intro-sentence that this is not possible and only left the references to the docs in my answer as a guide to others. What I actually meant to say was that the global variable scope is not automatically shared by the local scopes of the individual stages – but of course saying it's not possible was wrong/misleading. – Felix K. Mar 16 '21 at 14:28