variables substitution (or overriding) when extends jobs from gitlab templates unsing include

Question

Using gitlab ci, I have a repo where all my templates are created. For example, I have a sonar scanner job template named .sonar-scanner.yml:

sonar-analysis:
  stage: quality
  image:
    name: sonar-scanner-ci:latest
    entrypoint: [""]
  script:
    - sonar-scanner
      -D"sonar.projectKey=${SONAR_PROJECT_NAME}"
      -D"sonar.login=${SONAR_LOGIN}"
      -D"sonar.host.url=${SONAR_SERVER}"
      -D"sonar.projectVersion=${CI_COMMIT_SHORT_SHA}"
      -D"sonar.projectBaseDir=${CI_PROJECT_DIR}"

I have include this template as a project like this in main gitlab ci file:

include: 
  - project: 'organization/group/ci-template'
    ref: master
    file: '.sonar-scanner.yml'

So as you can understand I have a repo named ci-templates where all my templates are created. And in another repo, I extends using include these templates.

Finally, in a repo, when a new merge request is created, my job for sonar is running under another file in my project test/quality.yml:

sonar:
  stage: quality
  extends: 
    - sonar-analysis 
  allow_failure: true
  only:
    refs:
      - merge_requests

All is working well except the substitution or the overriding of my env var. Indeed of my template. I have many sonar server or project names. I would like to know how to override these variables SONAR_SERVER and SONAR_PROJECT_NAME when I extend the job from a template.

In my main .gitlab-ci.yml file I have a variables section declaration, and when I override these variables in, it works. But it's not really what I want. Using many stages and many micro service it is possible to reuse the same extending job in a different way. That I really want to do is to override these variables directly in the file test/quality.yml.

This, for example does not work:

sonar:
  stage: quality
  extends: 
    - sonar-analysis
  variables:
    SONAR_PROJECT_NAME: foo
    SONAR_SERVER: bar
  allow_failure: true
  only:
    refs:
      - merge_requests

This not work too:

variables:
  SONAR_PROJECT_NAME: foo
  SONAR_SERVER: bar

sonar:
  stage: quality
  extends: 
    - sonar-analysis 
  allow_failure: true
  only:
    refs:
      - merge_requests

What is the best way to make this work ?

I would have thought the first example would have worked... what happens if you try adding a `before_script` to just echo out the variables? If you have a top level `.gitlab-ci.yml` and then include the `test/quality.yml`, the top level `.gitlab-ci.yml` variables will override that of the `quality.yml` I believe... — Rekovni, Feb 10 '20 at 15:47
@Rekovni inded, if I put the var in the main `.gitlab-ci.yml`, the var will be overrided and it works well, but it's not that I need, I really need to find a way to make this in the `test.quality.yml`. If I use a `.before_script` nothing works. — french_dev, Feb 10 '20 at 15:59
Does `before_script: - echo $SONAR_PROJECT_NAME etc` not work in the `sonar` stage? Can you show how you have done the top level `.gitlab-ci` so far? — Rekovni, Feb 10 '20 at 16:03
That's weird because gitlab uses variables in their templates https://gitlab.com/gitlab-org/gitlab-foss/-/blob/master/lib/gitlab/ci/templates/Jobs/SAST.gitlab-ci.yml — Tofandel, Apr 02 '22 at 19:30

mike · Accepted Answer · 2023-05-06T01:53:30.960

After this question was asked in Feburary 2020, a new MR Use non-predefined variables inside CI include blocks has been merged into Gitlab 14.2, which resolves the issue for the overridden job.

The project that does the include can redefine the variables when extending a job:

include: 
  - project: 'organization/group/ci-template'
    ref: master
    file: '.sonar-scanner.yml'

sonar:
  stage: quality
  extends: 
    - sonar-analysis
  variables:
    SONAR_PROJECT_NAME: foo
    SONAR_SERVER: bar
  allow_failure: true

But in this case you probably want the job in the template to start with a dot .sonar-analysis instead of sonar-analysis to not create a real sonar-analysis job in the template (see hidden jobs).

Or you can also directly set the variables values (to redefine them) of an existing job in the project that does the include:

include: 
  - project: 'organization/group/ci-template'
    ref: master
    file: '.sonar-scanner.yml'

sonar-analysis:
  variables:
    SONAR_PROJECT_NAME: foo
    SONAR_SERVER: bar

I verified this with a test project, which includes a template from a peer test project, and when it runs, results in two jobs. This is the job output for the overriding job:

$ echo sonar.projectKey=${SONAR_PROJECT_NAME}
sonar.projectKey=foo
$ echo sonar.login=${SONAR_LOGIN}
sonar.login=bob
$ echo sonar.host.url=${SONAR_SERVER}
sonar.host.url=bar

variables substitution (or overriding) when extends jobs from gitlab templates unsing include

1 Answers1