Is there a way to dynamically choose whether a job is run in a Gitlab CI pipeline?

Question

I am trying to have one job check for a word being present in a config file and have that determine whether a subsequent trigger job occurs or not...

Like so...

stages:
   - check
   - trigger_pipeline

variables:
   - TRIGGER_JOB : "0"
   - CONFIG_FILE : "default.json"

check:
    stage: check
    script:
        - |
            if grep -q keyword "$CONFIG_FILE"; then
                TRIGGER_JOB="1"
            fi
        - echo "TRIGGER_JOB=${TRIGGER_JOB}" >> variables.env
    artifacts:
        reports:
            dotenv: "variables.env"

trigger_pipeline:
    stage: trigger_pipeline
    rules:
        - if: '$TRIGGER_JOB == "1"'
    trigger:
        project: downstream/project
        branch: staging
        strategy: depend
    needs: ["check"]

It seems like I've reached a limitation with GitLab because the trigger_pipeline job doesn't even get created due to the fact that the pipeline initializes with TRIGGER_JOB: "0" so it doesn't matter that I'm doing this check to trigger the pipeline later if the keyword was found.

Is there any way to dynamically decide if this trigger_pipeline job would be created or not?

I would just put it all in one job and trigger the downstream pipeline through the API, but then of course, I can't depend on the downstream status which is something I want as well (and isn't possible to do when triggering through the API from everything I've found in the docs).

Any advice would be appreciated. Thanks!

score 1 · Answer 1 · answered Feb 02 '22 at 03:54

1

The closest thing to what you're describing is dynamic child pipelines. That would allow you to create a pipeline configuration dynamically in one job, then run it.

generate-config:
  stage: build
  script: generate-ci-config > generated-config.yml
  artifacts:
    paths:
      - generated-config.yml


child-pipeline:
  stage: test
  trigger:
    include:
      - artifact: generated-config.yml
        job: generate-config

answered Feb 02 '22 at 03:54

sytech

29,298
3
45
86

is it possible to add retry: 1 to the child-pipeline – sathiya Feb 24 '23 at 15:57

score 0 · Answer 2 · answered Feb 03 '22 at 19:00

There is a way to dynamically choose whether or not to execute a gitlab job

In your case apply the following config

stages:
   - trigger_pipeline
   - check

variables:
   - CONFIG_FILE : "default.json"

check:
    stage: check
    script:
        - |
            if grep -q keyword "$CONFIG_FILE"; then
              curl -s --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.com/api/v4/projects/$CI_PROJECT_ID/pipelines/$CI_PIPELINE_ID/jobs" | jq '.[]'
              JOB_ID=$(curl -s --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.com/api/v4/projects/$CI_PROJECT_ID/pipelines/$CI_PIPELINE_ID/jobs" | jq '.[] | select(.name=="child-pipeline") | .id')
              curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.com/api/v4/projects/$CI_PROJECT_ID/jobs/$JOB_ID/play"
            fi

trigger_pipeline:
    stage: trigger_pipeline
    trigger:
        project: downstream/project
        branch: staging
        strategy: depend
    when: manual

The logic behind is that you configure your target job, in your case trigger_pipeline as manual

Change the stage ordering so that your target job is set up first

Then if your logic evaluates to true in your case

grep -q keyword "$CONFIG_FILE"

Then the check job basically executes the target job. By firstly identifying the target job id and finally executing by calling the play endpoint of the Gitlab API

Is there a way to dynamically choose whether a job is run in a Gitlab CI pipeline?

2 Answers2