Environment variable not injected into multi stage docker build on Azure Devops pipeline

Question

I have a docker file that I can build locally without issues, on Azure Devops the variable is not set properly. E.g. locally I can run a multi-stage docker build where artifacts are fetched from an Azure artifact repository with authorization. The authorization token can be set locally without issues. On the build pipeline I haven't been able to inject it properly.

The docker file:

FROM gradle:5.4.1-jdk8 AS build
ARG AZURE_ARTIFACTS_ENV_ACCESS_TOKEN
ENV AZURE_ARTIFACTS_ENV_ACCESS_TOKEN $AZURE_ARTIFACTS_ENV_ACCESS_TOKEN
COPY --chown=gradle:gradle . /home/gradle/src
WORKDIR /home/gradle/src
RUN gradle build --no-daemon

FROM java:8
COPY --from=build /home/gradle/src/build/libs/medquality-rest-service.jar medquality-rest-service.jar
ADD wait-for-it.sh /wait-for-it.sh
RUN chmod +x /wait-for-it.sh

ENTRYPOINT ["/wait-for-it.sh", \
            "${CORDA_NODE_URL}:${CORDA_NODE_PORT}", \
#            "--strict", \
            "--timeout=60", \
            "--", \
            "java", \
            "-jar", \
            "medquality-rest-service.jar", \
            "--config.rpc.host=${CORDA_NODE_URL}", \
            "--config.rpc.port=${CORDA_NODE_PORT}", \
            "--config.rpc.username=user1", \
            "--config.rpc.password=test"]

The command:

docker build --build-arg AZURE_ARTIFACTS_ENV_ACCESS_TOKEN .

It injects the token so the multistage build can fetch the artifacts.

Once I move to the Azure pipeline it will not inject the value, the pipeline:

trigger:
- master

pool:
  vmImage: 'ubuntu-latest'

steps:
- task: Gradle@2
  inputs:
    workingDirectory: ''
    gradleWrapperFile: 'gradlew'
    gradleOptions: '-Xmx3072m'
    javaHomeOption: 'JDKVersion'
    jdkVersionOption: '1.8'
    jdkArchitectureOption: 'x64'
    publishJUnitResults: true
    testResultsFiles: '**/TEST-*.xml'
    tasks: 'build publish'
  env:
      AZURE_ARTIFACTS_ENV_ACCESS_TOKEN: $(System.AccessToken)
- task: Docker@2
  inputs:
    containerRegistry: 'alysidia-container-registry'
    repository: 'medquality-rest-service'
    command: 'buildAndPush'
    arguments: --build-arg AZURE_ARTIFACTS_ENV_ACCESS_TOKEN=1234567
    Dockerfile: '**/Dockerfile'
  # env:
  #     AZURE_ARTIFACTS_ENV_ACCESS_TOKEN: $(System.AccessToken)

The 1st gradle task gets the variable injected properly but it seems I miss something related to the pipeline. The result currently is that the artifact PAT is not set and therefore the request is not authorized on the Docker task and its multi-stage build. E.g. even printing out all environment variables in the gradle script, AZURE_ARTIFACTS_ENV_ACCESS_TOKEN is not 1234567 but empty.

Update:

I've set hyphens on the arguments string, looked like a good candidate but no success, adding the RUN echo the value is not set:

arguments: '--build-arg AZURE_ARTIFACTS_ENV_ACCESS_TOKEN=$(System.AccessToken)'

The RUN section in the Dockerfile:

ARG AZURE_ARTIFACTS_ENV_ACCESS_TOKEN
RUN echo $AZURE_ARTIFACTS_ENV_ACCESS_TOKEN
ENV AZURE_ARTIFACTS_ENV_ACCESS_TOKEN $AZURE_ARTIFACTS_ENV_ACCESS_TOKEN

The output of the RUN command:

Step 3/21 : RUN echo $AZURE_ARTIFACTS_ENV_ACCESS_TOKEN
 ---> Running in 6791245d8990

Removing intermediate container 6791245d8990

Krzysztof Madej · Accepted Answer · 2020-11-23T15:43:05.410

1

I made a test for minimal example I mean for this Dockerfile

FROM alpine

ARG a_version
RUN echo $a_version

and this pipeline

steps:
- pwsh: ls 'stackoverflow/85-docker/'
- task: Docker@2
  inputs:
    containerRegistry: 'devopsmanual-acr'
    command: 'build'
    Dockerfile: 'stackoverflow/85-docker/DOCKERFILE'
    arguments: '--build-arg a_version=$(System.AccessToken)'

I got

2020-11-23T15:39:04.0075804Z Step 3/12 : RUN echo $a_version
2020-11-23T15:39:04.0228448Z  ---> Running in 45fc8efb4968
2020-11-23T15:39:04.3523862Z ***

which is correct because it detected secret and masked it.

If I run it for nor secret variable I have:

2020-11-23T15:42:10.0106169Z Step 3/12 : RUN echo $a_version
2020-11-23T15:42:10.0288192Z  ---> Running in a59622e31abb
2020-11-23T15:42:10.3746013Z SomeValue123

where SomeValue123 is value of my pipeline variable

edited Nov 23 '20 at 15:43

answered Nov 23 '20 at 12:18

Krzysztof Madej

32,704
10
78
107

Do you have only one Dockerfile in your repo? Can you try with something smaller? I mean it works and workout your Dockerfile it would be difficult to catch this edge case. Can you share your Dockerfile? – Krzysztof Madej Nov 23 '20 at 16:50
Yes in that particular repo there is only one Dockerfile, the Dockerfile is the first code snippet in the OP, I haven't changed it besides adding the "RUN echo" statement. I have one repo per service in general, those I push up to the registry and use them in a docker-compose locally. – 1174 Nov 23 '20 at 17:21
I've used your example, seems that "buildAndPush" does not work with arguments as expected, just using "build" as a command works, the variable is properly set. – 1174 Nov 23 '20 at 17:37
You can always split task into two separate. – Krzysztof Madej Nov 23 '20 at 17:47
Yes, I have to, the arguments property is not supported for buildAndPush as described: https://learn.microsoft.com/en-us/azure/devops/pipelines/tasks/build/docker?view=azure-devops RTFM :) Thank you very much for your help! – 1174 Nov 23 '20 at 17:48

Environment variable not injected into multi stage docker build on Azure Devops pipeline

1 Answers1

Linked