7

Background: We are looking for a solution on how to optimize our pipeline (former workflow).

Currently, we run quiet a few parallel deployments and tests which are spread on 2 builders, with 4 executors each.

The pipeline is triggered by a Git push, so subsequent pushes will trigger multiple builds. We have experimented with the stage concurrency: 1 option, which nicely blocks a step by a subsequent build, but will kick of when that specific stage is done.

Question(s):

I am not sure this is best practise, but It seems to me, it would be better to not execute the new build, until the previous one is done. (Reasoning from the fact that we have committed resources to it, and it should be allowed to finished, even if it's not the latest and greatest commit).

Q1: Is this even best practise?

Q2: how do we pre-empt the new triggert build, while still running the previous one? (I can imagine iterating through the builds of this job and stopping the new one...).

See the config of the first stage [1]

[1] first stage.. 

stage name: 'Checkout and build WAR'
node {
    def mvnHome = tool 'Maven 3.2.x'
    checkout([$class                           : 'GitSCM',
          poll                             : true,
          branches                         : [[name: '*/master']],
          doGenerateSubmoduleConfigurations: false,
          extensions                       : [[$class           : 'RelativeTargetDirectory',
                                               relativeTargetDir: 'checkout-directory']],
          submoduleCfg                     : [],
          userRemoteConfigs                : [[url: 'https://some.repo/repo.git']]])

// Archive the cloned repo.
stash name: 'src', includes: 'checkout-directory/war/src/, checkout-directory/war/pom.xml'

// Run without tests, do the unit and integration tests in a separate stage.
sh "${mvnHome}/bin/mvn -f checkout-directory clean install -DskipTests"

// Archive the application build.
stash name: 'war', includes: 'checkout-directory/war/target/*.war'
}
user1794565
  • 81
  • 1
  • 1
  • 3
  • See my answer here: http://stackoverflow.com/questions/36454130/how-do-i-prevent-two-pipeline-jenkins-jobs-of-the-same-type-to-run-in-parallel-o/38266233#38266233 – Vova Rozhkov Jul 08 '16 at 12:24

2 Answers2

3

From job's configuration you can set:

  • Execute concurrent builds if necessary
  • Quiet period

If set, a newly scheduled build waits for this many seconds before actually being built. This is useful for:

  • Collapsing multiple CVS change notification e-mails into one (some CVS changelog e-mail generation scripts generate multiple e-mails in quick succession when a commit spans across directories).
  • If your coding style is such that you commit one logical change in a few cvs/svn operations, then setting a longer quiet period would prevent Jenkins from building it prematurely and reporting a failure.
  • Throttling builds. If your Jenkins installation is too busy with too many builds, setting a longer quiet period can reduce the number of builds.

If not explicitly set at project-level, the system-wide default value is used.

As for jenkins-pipeline DSL this article answer you question:

By default, Pipeline builds can run concurrently. The stage command lets you mark certain sections of a build as being constrained by limited concurrency (or, later, unconstrained). Newer builds are always given priority when entering such a throttled stage; older builds will simply exit early if they are preëmpted.

luka5z
  • 7,525
  • 6
  • 29
  • 52
  • Ah.. thx. Of course the regular job config can pre-empt concurrency. I was reasoning from the DSL script, which apparently is not capable to do this. The concurrency:1 option on a stage, doesn't cut it, because when that specific stage is done, it will enter concurrent build. – user1794565 Apr 21 '16 at 09:02
  • As of Aug 17, the `stage` commands concurrency parameter is deprecated and shouldn't be used anymore. An alternative approach is using the `milestone` step, which does not hinder builds to start but to wait for the previous build. – Jazzschmidt Sep 04 '17 at 09:16
  • Pardon, I meant to say the `lock` step. – Jazzschmidt Sep 04 '17 at 09:30
1

Using the Jobs configuration through properties seems to be the cleanest way to go.

See https://stackoverflow.com/a/43963315/1756183

dag
  • 1,133
  • 12
  • 25