Limit Jenkins concurrent build only for specific branches

Question

I have been looking for how to do it for a while and I am starting to think this is not possible. Every question similar to this one seems to be more related on limiting some nodes running or the entire pipeline.

We have a multibranch pipeline in Jenkins that handles PR builds, staging testing and production release. Each of the three are handled via the when option in each stage because the difference is whether to run or not some stages.

While I would like to be able to have concurrent builds for PRs because we can have many opened that need updating, we want to limit the number of concurrent executions for the staging test because we need to do one staging run at the time to be able to control it properly.

In a previous version we had several different pipelines handling each scenario but we are updating it to have only one multibranch pipeline and run one scenario or another depending on the branch that is being considered. In this previous version we used the disableConcurrentBuilds() for the staging test build and it worked perfectly but with this option we now see that the entire pipeline runs only one build at the time which makes it extremely slow to get the PR builds done. If a PR build is running, since there cannot be concurrent runs, the rest of the PRs are just waiting for it to finish so the next can start running.

Is there a way we can limit the number of concurrent build only for some branches or per branch? This will allow us to disable this only for the branch where we run staging tests while still being able to run concurrently for the PRs that are open.

Thanks!

score 7 · Accepted Answer · answered Nov 09 '20 at 15:18

7

The Jenkins UI doesn't allow you to specify this on a per-branch basis, but it is possible if you configure the job via your jenkinsfile

if(env.BRANCH_NAME.startsWith("develop")){
  properties([disableConcurrentBuilds()])
} else{
  properties([])
}

If this is too coarse for you because you don't want multiple branches reaching a test or deployment stage at the same time you can use locks and milestones To allow concurrency where possible but only allow a single pipeline at a time through a given stage

answered Nov 09 '20 at 15:18

stewartie4

190
1
8

This looks great! I didn't intend to configure it from the UI anyway hehe. I did not know about locks and that sounds exactly like what I would love to do! I would love some parts of the pipelines to run concurrently and limit to only one at the time for some parts of it. – Juanpe Araque Nov 10 '20 at 09:06

score 0 · Answer 2 · answered Nov 09 '20 at 15:29

While the previous answer is certainly worth investigating, you may also fiddle with executors and labels to achieve what you want.

Here's an example of what I mean:

    stage('Run Staging tests') {
        agent { node { label 'staging' }}
        when {
            expression { 
                return someCondition() 
                // analyse a branch and return true if need to run
            }
            beforeAgent true // evaluate the condition on master
        }
        steps {
            ...

If there is only one node with the label staging and that node is configured with one executor, then only one job could be running on that node at any time. The other jobs wanting to use that label (presumably to run staging tests) would wait until the previous job is done. The other jobs not planning to use the label (because the condition is not met) would simply run as usual on other nodes without any waiting whatsoever.

This looks actually like a very bullet proof approach. Sadly I won't be able to follow this route because I do not have control over the executors :S this is a centralised Jenkins instance for the entire company with a rather complex configuration to suit every team. While I could request something like this seems like going too far. Thanks! — Juanpe Araque, Nov 10 '20 at 09:11

Limit Jenkins concurrent build only for specific branches

2 Answers2

Linked