10

I am trying to create a automation pipeline for data load. I have a scenario as explained below:

stages
 - stage1
- stage2

job1:
  stage: stage1
  script:
  - echo "stage 1 job 1"


job2:
  stage: stage1
  script:
   - echo "stage 1 job 2"

job3:
  stage: stage1
  script:
   - echo "stage 1 job 3"

job4:
  stage: stage1
  script:
   - echo "stage 1 job 4"

I want to run the job1 and job2 parallel in the same stage. So, after Job1 and job2 success

  • job1 will invoke/trigger the job3. that means job3 will start automatically when job1 successes
  • job2 will invoke/trigger the job4 that means job4 will start automatically when job2 successes

I am writing the pipeline in the .gitlab-ci.yml.

Can anyone help me to implement this?

Vadim Kotov
  • 8,084
  • 8
  • 48
  • 62
VIPIN PAL
  • 111
  • 1
  • 1
  • 3

1 Answers1

17

Strict implementation of your requirements is not possible (according to my knowledge), the jobs 3 and 4 would need to be in a separate stage (although support for putting them in the same stage is planned). To be clear: the other functional requirements can be fulfilled, i.e:

  • job1 and job2 start in parallel
  • job1 will trigger the job3 (immediately, without waiting for the job2 to finish)
  • job2 will trigger the job4 (immediately, without waiting for the job1 to finish)

The key is using needs keyword to convert the pipieline to a directed acyclic graph:

stages:
    - stage-1
    - stage-2

job-1:
    stage: stage-1
    needs: []
    script: 
      - echo "job-1 started"
      - sleep 5
      - echo "job-1 done"

job-2:
    stage: stage-1
    needs: []
    script: 
      - echo "job-2 started"
      - sleep 60
      - echo "job-2 done"

job-3:
    stage: stage-2
    needs: [job-1]
    script: 
      - echo "job-3 started"
      - sleep 5
      - echo "job-3 done"

job-4:
    stage: stage-2
    needs: [job-2]
    script: 
      - echo "job-4 started"
      - sleep 5
      - echo "job-4 done"

pipeline

As you can see on the screenshot, the job 3 is started, even though the job 2 is still running.

Mafor
  • 9,668
  • 2
  • 21
  • 36
  • Hi @mafor thank you for your response. And it worked for me first time. But later when I try to implement this it give me a error saying "found error in your .gitlab-ci.yml: job-3: needs 'job-1' ". I have also test via CI lint there is no syntax error. I don't know why this error is coming. – VIPIN PAL Oct 14 '20 at 08:07
  • @VIPINPAL it might be related to the specifics of your jobs, e.g. usage of the `only` keyword. See https://gitlab.com/gitlab-org/gitlab/-/issues/207225 – Mafor Oct 14 '20 at 11:00