How can I conditionally use a container on Azure Pipelines?

Question

I have an AzP job with a matrix where one needs to run in a Container (old compiler test) How can I conditionally run the jop inside the container? I tried the below but that doesn't seem to work, i.e. the container is never used although I'm pretty sure I followed all instructions from the docs

stages:
  - stage: Test
    jobs:
      - job: 'Linux'
        strategy:
          matrix:
            GCC_10:
              CXX: g++-10
              VM_IMAGE: ubuntu-20.04
            GCC_9:
              CXX: g++-9
              VM_IMAGE: ubuntu-20.04
            Clang_3_7:
              CXX: clang++-3.7
              VM_IMAGE: ubuntu-20.04
              CONTAINER: ubuntu:16.04
        pool:
          vmImage: $(VM_IMAGE)
        ${{ if variables['CONTAINER'] }}:
          container:
            image: $[ variables['CONTAINER'] ]
            options:  "--name ci-container -v /usr/bin/docker:/tmp/docker:ro"

score 1 · Accepted Answer · answered Nov 20 '21 at 13:36

You can't do this with variables from matrix, since template expressions (${{ }}) are rendered very early and they don't have access to those variables. See this page to get an idea of how pipeline is processed: Pipeline run sequence

1. First, expand templates and evaluate template expressions.

...

4. For each job selected to run, expand multi-configs (strategy: matrix or strategy: parallel in YAML) into multiple runtime jobs.

But you can achieve desired result by using runtime parameters, which are available to the template expressions:

parameters:
  - name: matrix
    type: object
    default:
      GCC_10:
        CXX: g++-10
        VM_IMAGE: ubuntu-20.04
      GCC_9:
        CXX: g++-9
        VM_IMAGE: ubuntu-20.04
      Clang_3_7:
        CXX: clang++-3.7
        VM_IMAGE: ubuntu-20.04
        CONTAINER: ubuntu:16.04

stages:
  - stage: Test
    jobs:
      - ${{ each item in parameters.matrix }}:
          - job: Linux_${{ item.Key }}
            pool:
              vmImage: ${{ item.Value.VM_IMAGE }}
            ${{ if item.Value.CONTAINER }}:
              container:
                image: ${{ item.Value.CONTAINER }}
                options: --name ci-container -v /usr/bin/docker:/tmp/docker:ro
            steps:
              - bash: echo CXX = ${{ item.Value.CXX }}

This will be rendered to

parameters:
- name: matrix
  type: object
  default:
    GCC_10:
      CXX: g++-10
      VM_IMAGE: ubuntu-20.04
    GCC_9:
      CXX: g++-9
      VM_IMAGE: ubuntu-20.04
    Clang_3_7:
      CXX: clang++-3.7
      VM_IMAGE: ubuntu-20.04
      CONTAINER: ubuntu:16.04
stages:
- stage: Test
  jobs:
  - job: Linux_GCC_10
    pool:
      vmImage: ubuntu-20.04
    steps:
    - task: Bash@3
      inputs:
        targetType: inline
        script: echo CXX = g++-10
  - job: Linux_GCC_9
    pool:
      vmImage: ubuntu-20.04
    steps:
    - task: Bash@3
      inputs:
        targetType: inline
        script: echo CXX = g++-9
  - job: Linux_Clang_3_7
    pool:
      vmImage: ubuntu-20.04
    container:
      image: ubuntu:16.04
      options: --name ci-container -v /usr/bin/docker:/tmp/docker:ro
    steps:
    - task: Bash@3
      inputs:
        targetType: inline
        script: echo CXX = clang++-3.7

Awesome, that work very well! I did some minor changes to use a list instead of objects in the parameter and auto-create the job-name if not explicitely specified via a `name: xxx` key in the list-item. But yeah, trivial changes based on your answer — Flamefire, Nov 21 '21 at 11:32

How can I conditionally use a container on Azure Pipelines?

1 Answers1