16

I am new to GitLab and facing a problem where if I trigger two pipelines at the same time on same gitlab-runner, they both run in parallel and results in failure. What I want is to limit the run to one pipeline at a time and others in queue.

I have set the concurrent = 1 in config.toml and restarted the runner but it didn't help. My ultimate goal is to prevent multi-pipeline run on the runner.

Thanks.

Shubh Rocks Goel
  • 519
  • 1
  • 6
  • 17
  • Does this answer your question? [How to force GitLab to run a complete pipeline before starting a new one?](https://stackoverflow.com/questions/49536377/how-to-force-gitlab-to-run-a-complete-pipeline-before-starting-a-new-one) – vctls May 12 '20 at 16:41

4 Answers4

11

Set resource_group in the Job, and give a unique name for all other tasks that should be blocked.

Example from the documentation:

deploy-to-production:
  script: deploy
  resource_group: production
phihag
  • 278,196
  • 72
  • 453
  • 469
  • Thanks. This solved it for me. There is also a way to control concurrency when your limited pipeline is a child pipeline: https://docs.gitlab.com/ee/ci/resource_groups/#pipeline-level-concurrency-control-with-cross-projectparent-child-pipelines – Reboot_87 Aug 15 '22 at 14:59
  • 3
    From my reading, this will not work in some cases. If you have a pipeline with multiple jobs and you set all of them to the same resource_group, GitLab isn't making sure that one pipeline finished before the other starts. Instead, it's making sure that only one job runs at a time. When one job finishes it can pick any job from either pipeline to run next. – Todd Walton Oct 17 '22 at 21:29
  • @ToddWalton The dependencies of jobs can be configured with `needs`. Sounds like you want that, and a multi-stage pipeline. This question and answer is applicable to pipelines with only one single step. – phihag Oct 17 '22 at 22:21
  • 1
    The question doesn't say it's a one-job pipeline. Also, using `needs` would not prevent multiple pipelines from running at a time. `needs` only orders jobs in a single pipeline. GitLab could still kick off jobs in a second pipeline after a job in a first pipeline finishes, irrespective of any `needs` keywords. It's funky, thinking through how the pipelines work, because there's so many configurable parts. – Todd Walton Oct 20 '22 at 13:48
4

The answer by @phihag works also for pipelines with multiple jobs. The only thing missing is a specific configuration of the resource_group:
Set process_mode=oldest_first 

curl --request PUT --data "process_mode=oldest_first" \
     --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/1/resource_groups/production"
Nazar
  • 91
  • 4
0

As others have mentioned resource_groups are the key feature that you need.

Setting concurrent to 1 within config.toml is not enough as you already mentioned. Even though Gitlab will execute one job after the another, it will pick up jobs in an unordered manner from different pipelines and that may be a problem as you already mentioned, especially when you use a shell executor and jobs from different pipelines work on the same build.

What you could do is for example give all jobs the same resource_group and then — now that is important — make an API call and set the process_mode of this resource_group to oldest_first, because the default is unordered.

Now the second pipeline will not start and wait for the first pipeline to finish and therefore, one pipeline runs at a time, exactly what you want.

Ini
  • 548
  • 7
  • 19
-7

Set the limit keyword in the runners section of your configuration to 1.

limit :

Limit how many jobs can be handled concurrently by this token. 0 (default) simply means don’t limit

and restart you runner

Nicolas Pepinster
  • 5,413
  • 2
  • 30
  • 48
  • 9
    As stated by the doc, this limits concurrency of jobs, not pipelines. Runner can still start the first job of a second pipeline before doing all jobs of the first pipeline. Dig a little more and I think you'll always get to this issue, that's been postponed for ages: https://gitlab.com/gitlab-org/gitlab/-/issues/15536 – vctls May 12 '20 at 16:24
  • 1
    The actual, most recent and relevant issue on gitlab.com: https://gitlab.com/gitlab-org/gitlab/-/issues/202186 – vctls May 12 '20 at 17:01