GitLab CI custom variable not recognized in rules

Question

When defining a variable in the variables block which uses a predefined CI variable, it cannot be referenced in the rules blocks. Here, only job_1 gets executed:

variables:
  PRODUCTION_BRANCH: $CI_DEFAULT_BRANCH

stages:
  - stage_1

job_1:
  stage: stage_1
  script:
    - export
  rules:
    - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'

job_2:
  stage: stage_1
  script:
    - export
  rules:
    - if: '$CI_COMMIT_BRANCH == $PRODUCTION_BRANCH'

But when hardcoding the value for PRODUCTION_BRANCH it works as expected. Here, both jobs get executed:

variables:
  PRODUCTION_BRANCH: "master"

stages:
  - stage_1

job_1:
  stage: stage_1
  script:
    - export
  rules:
    - if: '$CI_COMMIT_BRANCH == $CI_DEFAULT_BRANCH'

job_2:
  stage: stage_1
  script:
    - export
  rules:
    - if: '$CI_COMMIT_BRANCH == $PRODUCTION_BRANCH'

When looking into the output of the jobs (since export logs the env-variables), in both cases there is the variable PRODUCTION_BRANCH with the correct value master.
Why does it behave like that and is there a fix / workaround?

Answer 1

Update 2022-09:

It appears that gitlab has fixed some issues related to this.

I haven't had a chance to re-test yet whether this is completely resolved issue or if there are still some pitfalls to be aware of.

Just be aware that the below answer may no longer be correct/current.

---

gitlab does not do variable expansion of the variables section when evaluating the yaml and determining what to run. Variable interpolation does work within the runner though.

So you have to be careful if you are using variable interpolation when defining custom variables. Are they for gitlab conditions (won't work) or are they for use in runner only (after gitlab has decided to run the job...this works)?

This is confusing and I couldn't find any warning/reference to this high-probability pitfall in the docs...but maybe I missed it.

So in your if: you can use

• $CI_* variables
  • (except for a few ENVIRONMENT ones ...see description column for rules in the linked where variables can be used table.)
• custom variables you have defined globally or within a job

  • variables:
        PRODUCTION_BRANCH: "main"
    

But if your if: depends on custom variables (ie $PRODUCTION_BRANCH) that themselves require variable interpolation...

variables:
    PRODUCTION_BRANCH: $CI_DEFAULT_BRANCH

...it won't work. $PRODUCTION_BRANCH will have the value you expect within the RUNNER (once the job is started) but gitlab doesn't do the interpolation when parsing the yaml and deciding which jobs to run.

In your case, simplest thing is just to use the $CI_DEFAULT_BRANCH variable directly in your if:.

Example .gitlab-ci.yml demonstrating this...

You can drop this in a new empty repo. Just update REPO_NAMESPACE, REPO_NAME and ONLY_DEPLOY_BRANCH to match the current repo/branch.

jobs:

• debug - runs and outputs vars showing all the values you expect (in a runner context)
• deploy1 - runs
• deploy2 - does not run
• deploy3 - runs (shows the variables are definitely not interpolated on the gitlab side).

variables:
  REPO_NAMESPACE: "my-group/my-subgroup"
  REPO_NAME: "my-project"
  ONLY_DEPLOY_BRANCH: "main"
  ONLY_DEPLOY_FROM: "${REPO_NAMESPACE}/${REPO_NAME}"
  ONLY_DEPLOY_FROM2: "my-group/my-subgroup/my-project"



stages:
  - debug
  - deploy

# this runs
# variables all have expected values (in runner scope) and you would expect both deploy* jobs to run
debug:
  stage: debug
  script:
    - export  

# this runs
deploy1:
  stage: deploy
  image: alpine:latest
  rules:
    - if: $CI_PROJECT_PATH == $ONLY_DEPLOY_FROM2 && $CI_COMMIT_REF_SLUG == $ONLY_DEPLOY_BRANCH
      when: always
  script:
    - echo "Hello world!"

# this doesn't run.  
# Only difference to deploy1 job is using $ONLY_DEPLOY_FROM instead of $ONLY_DEPLOY_FROM2
# conclusion... gitlab must not do variable expansion of the variables section when determining which jobs run.
deploy2:
  stage: deploy
  image: alpine:latest
  rules:
    - if: $CI_PROJECT_PATH == $ONLY_DEPLOY_FROM && $CI_COMMIT_REF_SLUG == $ONLY_DEPLOY_BRANCH
      when: always
  script:
    - echo "Hello world!"

# this runs which confirms that $ONLY_DEPLOY_FROM has not had CI variables replaced.
deploy3:
  stage: deploy
  image: alpine:latest
  rules:
    - if: $ONLY_DEPLOY_FROM == "$REPO_NAMESPACE/${REPO_NAME}" && $CI_COMMIT_REF_SLUG == $ONLY_DEPLOY_BRANCH
      when: always
  script:
    - echo "Hello world!"
    
    

You can use the ci linter (https://gitlab.com/${CI_PROJECT_PATH}/-/ci/lint) to play with this. It is a good tool in general for troubleshooting problems without making a thousand commits in a row to .gitlab-ci.yml and burning through shared runner hours.

Open issue

Gitlab has ~40k open issues. Here are the relevant tickets though...

Supporting variable expansion for runner (resolved)
Supporting the gitlab side variable expansion -- this problem. still open issue. - closed Apr 27, 2022

Answer 2

Just put the value in quotes:

variables:
  PRODUCTION_BRANCH: "$CI_DEFAULT_BRANCH"