Is there a way to parametrize/dynamically set variable group names in Azure DevOps Pipelines YAML?

Question

I have a nested Azure DevOps YAML pipeline:

---
name: Some Release Pipeline

trigger: none

variables:
  - group: "DEV VARIABLE GROUP" # This is the environment variable library

stages:
  - stage: Stage1
    displayName: "Stage 1"
    dependsOn: []
    jobs:
      - template: /pipelines/pipeline_templates/sometemplate.yml

What I would like to do is reuse this release pipeline for any environment. Ideally I would set a pipeline variable "group-name" and then assign it to the group. Something like this:

---
name: Some Release Pipeline

trigger: none

variables:
 - group: "$(group-name)" # This is the environment variable library

stages:
 - stage: Stage1
    displayName: "Stage 1"
    dependsOn: []
    jobs:
      - template: /pipelines/pipeline_templates/sometemplate.yml

However, this doesn't seem to work. Desperately, I've tried a variety of approaches:

• Using ${{ group-name }}
• I've tried to pass the group-name as a parameter using:

    jobs:
      - template: /pipelines/pipeline_templates/sometemplate.yml
        parameters:
          variablegroup: $(group-name)

and then setting it in the sometemplate.yml in the job. eg:

jobs:
  - job: Job1
    variables:
      - group: ${{ parameters.variablegroup }}

However, this didn't work neither.

• I've tried using insertion ({{ insert }}) as suggested here. However, either I don't know how to use insertion properly, or that doesn't work either as I was always encountering some form of validation errors.

According to this, and this, and this, and this it doesn't seem possible.

I am wondering if anyone has found a solution to this yet (other then doing a really messy workaround of calling the DevOps REST API)?

Answer 1

Your intuition of passing the group name as a parameter to the template was correct. I was able to get this working:

Template file passing-variable-groups.yml

parameters:
- name: deploymentVariableLibraries
  type: object
  default:
    dev: ''
    qa: ''
    prod: ''

jobs:
- job: TestDev
  variables:
    - group: ${{parameters.deploymentVariableLibraries.dev}}
  steps:
    - script: echo "$(whichEnvironment)"
- job: TestQa
  variables:
    - group: ${{parameters.deploymentVariableLibraries.qa}}
  steps:
    - script: echo "$(whichEnvironment)"
- job: TestProd
  variables:
    - group: ${{parameters.deploymentVariableLibraries.prod}}
  steps:
    - script: echo "$(whichEnvironment)"

Pipeline

trigger: none

resources:
  repositories:
    - repository: templates
      type: git
      name: c4ePipelineExamples-Templates

jobs:
- template: passing-variable-groups.yml@templates
  parameters:
    deploymentVariableLibraries:
      dev: 'test-dev'
      qa: 'test-qa'
      prod: 'test-prod'

My output from each job is DEV, QA, PROD respectively (the value of whichEnvironment in each test variable library).

Answer 2

Following worked for me without any templates

variables:
- group: variables-${{variables['Build.SourceBranchName']}}

Answer 3

Have you tried injecting the variables mapping as a parameter.

Parameters are not limited to scalar strings. As long as the place where the parameter expands expects a mapping, the parameter can be a mapping. Likewise, sequences can be passed where sequences are expected.

# sometemplate.yml
parameters:
  variables: {}

jobs:
- job: build
  variables: ${{ parameters.variables }}

# somepipeline.yml
name: Some Release Pipeline

trigger: none

stages:
 - stage: Stage1
    displayName: "Stage 1"
    dependsOn: []
    jobs:
      - template: sometemplate.yml
        parameters:
          variables: 
            group: "DEV VARIABLE GROUP"

I've used parameters to inject task sets as before/after actions to other tasks in a template, but I haven't used it to inject a mapping. I would think the area that this particular implementation would get sticky is in the need to use name/value syntax. I imagine if you're passing a group mapping into a job, anything else defining variables would need to use the extended syntax.

If you use both variables and variable groups, you'll have to use name/value syntax for the individual (non-grouped) variables:

Also, you may need to be careful as to when you're using runtime $() vs "compile" or expansion time ${{ }} syntax. Working with variables you may want to use expansion time references where possible.

Answer 4

played around a bit based on @antonpinchuk answer, and this seemed to do the trick for my scenario:

variables:
  - ${{ if eq(variables['Build.SourceBranch'], 'refs/heads/develop') }}:
    - group: vars-dev
  - ${{ if eq(variables['Build.SourceBranch'], 'refs/heads/main') }}:
    - group: vars-prod

The parameter approach also works very well, you could even do a dry approach for your example:

jobs:
- ${{ each varGroup in parameters.deploymentVariableLibraries }}:
  - job: ${{ varGroup }}
    variables:
    - group: ${{ varGroup }}