How to pass array variables to parallel matrix in GitLab CI pipeline?

Question

Trying to configure parallel testing. Is there some way to set up a variable as an array and use it in matrix? For example:

stages:
- test

variables:
  SUITES: $SUITES

test:
  stage: test
  image: $CI_REGISTRY_IMAGE
  parallel:
    matrix:
      - SUITE: [$SUITES]
  script:
    - pytest -m ${SUITE} --json-report --json-report-file=${SUITE}_report.json
  artifacts:
    when: always
    paths:
      - "${SUITE}_report.json"
    expire_in: "1 day"

The case is to run jobs in parallel with suite and artifacts for each job. Maybe, I'm looking the wrong place?

score 0 · Answer 1 · answered Jan 25 '23 at 21:22

I have a similar question about the matrix feature. I have a pipeline template that can build multiple images of a "base" docker image, where each image differs in the version of the tool. For example, I want to build custom "base" .NET images for .NET 3.1, 5.0, and 6.1.

Previously I was declaring a variable: VERSIONS_TO_BUILD: "3.1 5.0 6.0"

and then looping through that list (eg: foreach ver in VERSION_TO_BUILD, run docker build).

I am also scanning the resulting containers. So, multiple jobs would have the same matrix list.

I just discovered this matrix functionality. I realize I can setup my job as such:

build:
  parallel:
    matrix:
      - VERSION: 3.1
      - VERSION: 5.0
      - VERSION: 6.0

# repeat for scan job

As mentioned, I am using a template so the same pipeline can be used for .NET, Node, Java, Maven, etc. What I am hoping to do is to include the template, then define the versions I'm using for that repo, then re-use it.

include:
  - base_image_pipeline.yml

variables:
  VERSIONS:
    - "3.1"
    - "5.0"
    - "6.0"

build:
 parallel:
   matrix:
    - $VERSIONS

scan:
 parallel:
   matrix:
    - $VERSIONS

I have a feeling the !reference keyword might be the best option, but would like other inputs.

Thanks!

score -1 · Answer 2 · answered Oct 17 '22 at 18:54

-1

See the GitLab docs on the parallel:matrix keyword. The whole idea is to set up multiple variable definitions which will each be run in parallel. Each list element in parallel will be one job specification; list elements should be dictionaries specifying the variables to be set in each job.

In your case:

test:
  stage: test
  image: $CI_REGISTRY_IMAGE
  parallel:
    matrix:
    - SUITE: endpoints
    - SUITE: smoke
    - SUITE: auth
  script:
    - pytest -m ${SUITE} --json-report --json-report-file=${SUITE}_report.json
  artifacts:
    when: always
    paths:
      - "${SUITE}_report.json"
    expire_in: "1 day"

answered Oct 17 '22 at 18:54

Michael Delgado

13,789
3
29
54

Thank you for your reply! I actually need to pass SUITE Variable as an array like this: SUITES: $SUITE I was incorrect in my question, sorry – Dmitriy Klepikov Oct 18 '22 at 14:28
why? can you please modify your question to include the full context, including how/when the value of SUITE will be set? is SUITE a project or group variable, or is it being passed in from a trigger project, etc? the context matters – Michael Delgado Oct 18 '22 at 18:17
I have a similar use case where the variable of the matrix (here SUITE) should be read from a variable $SUITES set from a former job in my pipeline. \ So I have also something like \ - SUITE: [$SUITES] \ but my job do NOT explode the strings there. – EricBDev Jan 24 '23 at 19:03
if you have a new question please create a new post, don't add it as a comment to this answer. see [ask] – Michael Delgado Jan 24 '23 at 20:50

How to pass array variables to parallel matrix in GitLab CI pipeline?

2 Answers2