Adds needs relations to GitLab CI yaml but got an error: the job was not added to the pipeline

Question

I am trying to add needs between jobs in the Gitlab CI yaml configuration file.


stages:
  - build
  - test
  - package
  - deploy

maven-build:
  stage: build
  only:
    - merge_requests
    - master
    - branches
  ...
test:
  stage: test
  needs: [ "maven-build" ]
  only:
    - merge_requests
    - master
  ...
docker-build:
  stage: package
  needs: [ "test" ]
  only:
    - master
  ...
deploy-stage:
  stage: deploy
  needs: [ "docker-build" ]
  only:
    - master
  ...
deploy-prod:
  stage: deploy
  needs: [ "docker-build" ]
  only:
    - master
  when: manual
  ...

I have used the GitLab CI online lint tools to check my syntax, it is correct.

But when I pushed the codes, it always complains:


    'test' job needs 'maven-build' job
    but it was not added to the pipeline

You can also test your .gitlab-ci.yml in CI Lint

The GitLab CI did not run at all.

Update: Finally I made it. I think the needs position is sensitive, move all needs under the stage, it works. My original scripts included some other configuration between them.

Do you have any `rules` or `only`/`except`on your build job which prevents it from being created in certain cases? — danielnelz, Apr 22 '21 at 10:19
@danielnelz Not used rules,but used `only` in the jobs. The original post is updated. — Hantsy, Apr 22 '21 at 14:16
I create a simple test just echo as script with needs dependency. It works fine. Can you try comments complex script and test out. — Rockie Yang, Apr 27 '21 at 04:48

score 38 · Answer 1 · answered May 19 '21 at 20:26

38

CI-jobs that depend on each other need to have the same limitations!

In your case that would mean to share the same only targets:

stages:
  - build
  - test

maven-build:
  stage: build
  only:
    - merge_requests
    - master
    - branches

test:
  stage: test
  needs: [ "maven-build" ]
  only:
    - merge_requests
    - master
    - branches

that should work from my experience^^

answered May 19 '21 at 20:26

dniwdeus

491
3
3

2

I already made it work. I think the `needs` position is sensitive, move all `needs` under the `stage`, it works. My original scripts included some other configuration between them. – Hantsy May 20 '21 at 01:38
Well in my case it was exactly this. Jobs need to share same limitations. Thanks! – Steven Rosato May 31 '21 at 21:36
1

> In GitLab 13.9 and older, if needs: refers to a job that might not be added to a pipeline because of only, except, or rules, the pipeline might fail to create. https://docs.gitlab.com/ce/ci/yaml/#requirements-and-limitations stupid me, i thought this was fixed. – Salz Jun 08 '21 at 10:48
Re-ordering the keyword 'needs' helped. Thank you. – usert4jju7 Jul 03 '21 at 04:13
1

checking for rules/only is a good point. Gitlab-CI Pipeline Editor showed to me, that the pipeline is valid even though the rules didn't match – takethefake Aug 16 '22 at 06:48
Ordering "needs" after "stage" does not solve the problem. However, having the same conditions (rules, only, except, ...) does solve the problem. – Romain Vincent Dec 08 '22 at 18:13

score 8 · Answer 2 · answered Aug 23 '21 at 20:53

8

Finally I made it. I think the needs position is sensitive, move all needs under the stage, it works

Actually... that might no longer be the case with GitLab 14.2 (August 2021):

Stageless pipelines

Using the needs keyword in your pipeline configuration helps to reduce cycle times by ignoring stage ordering and running jobs without waiting for others to complete.

Previously, needs could only be used between jobs on different stages.

In this release, we’ve removed this limitation so you can define a needs relationship between any job you want.

As a result, you can now create a complete CI/CD pipeline without using stages by including needs in every job to implicitly configure the execution order.
This lets you define a less verbose pipeline that takes less time to create and can run even faster.

See Documentation and Issue.

answered Aug 23 '21 at 20:53

VonC

1,262,500
529
4,410
5,250

Hi @VonC, Just a question not related to this thread. Is it possible to use a stage name instead of the job name to make a job dependent on another job? That is, instead of using "build-job" can I by any chance use "build" to make a job depend on job(s) with stage "build"? It doesn't work in dependencies but is there anything else with which it might work? Thanks. – Vibhav Chaddha Sep 16 '21 at 13:43
1

@VibhavChaddha Not sure, I have not tested that use case. Let me know if you ask that as a separate question, for me to follow along. – VonC Sep 16 '21 at 14:16
Hi, @VonC I have posted a new question regarding my concern. https://stackoverflow.com/questions/69278440/can-we-use-a-stage-name-instead-of-a-job-name-to-make-a-job-dependent-on-another – Vibhav Chaddha Sep 22 '21 at 05:04

score 0 · Answer 3 · answered Apr 27 '22 at 13:38

0

The rule in both jobs should be that same or otherwise GitLab cannot create job dependency between the jobs when the trigger rule is different.

answered Apr 27 '22 at 13:38

ayakout

49
7

score 0 · Answer 4 · answered Oct 06 '22 at 20:42

I don't know why, but if the jobs are in different stages (as in my case), you have to define the jobs that will be done later with "." at the start.

Another interesting thing is GitLab's own CI/CD Lint online text editor does not complain there is an error. So you have to start the pipeline to see the error.

Below, notice the "." in ".success_notification" and ".failure_notification"

stages:
- prepare
- build_and_test
- deploy
- notification

#SOME CODE

build-StandaloneWindows64:
  <<: *build
  image: $IMAGE:$UNITY_VERSION-windows-mono-$IMAGE_VERSION
  variables:
    BUILD_TARGET: StandaloneWindows64

.success_notification:
  needs:
    - job: "build-StandaloneWindows64"
      artifacts: true
  stage: notification
  script:
    - wget https://raw.githubusercontent.com/DiscordHooks/gitlab-ci-discord-webhook/master/send.sh
    - chmod +x send.sh
    - ./send.sh success $WEBHOOK_URL
  when: on_success

.failure_notification:
  needs:
    - job: "build-StandaloneWindows64"
      artifacts: true
  stage: notification
  script:
    - wget https://raw.githubusercontent.com/DiscordHooks/gitlab-ci-discord-webhook/master/send.sh
    - chmod +x send.sh
    - ./send.sh failure $WEBHOOK_URL
  when: on_failure

#SOME CODE

score 0 · Answer 5 · answered Jul 05 '23 at 07:36

0

I had a similar problem. It turned out that the previous stage did not work out because of the only block, so the stage depending on it could not work out either.

answered Jul 05 '23 at 07:36

Alex Sidler

11
2

Adds needs relations to GitLab CI yaml but got an error: the job was not added to the pipeline

5 Answers5

Stageless pipelines