0

We have an existing Jenkins install that is testing firmware running on an embedded tart. The multi-stage pipeline looks something like: Checkout -> Build -> Download -> Smoke tests -> Unit tests. This is working great, except it takes 9 hours to run the pipeline. To speed things up and also to test different target variants we have added 3 more targets to the system (UUT#1, #2, and so on).

My question is, what is the most straightforward way to allow the parallelization happen while also restricting the suites to UUTs with specific properties. For example, our Unit tests contain about 10 different suites (suite1 suite2 and so on), and what I’d like to do is spread those out amongst the 4 UUTs (thus having 4 suites running at a time) but restrict the execution this way:

  • Suite1 can only run on a UUT that has ‘USB’
  • Suite2 can only run on a UUT that has ‘LCD-display’
  • Suite3 can run anywhere

.. and so on, then my UUTs might have properties like:

  • UUT#1 ‘USB LCD-display’
  • UUT#2 ‘Ethernet’
  • UUT#3 ‘RS-232 USB’

Etc.

Reading about agents, it seems that a label on an agent may allow this, but agents seem to carry a lot of overhead and I’m not sure if they’re appropriate.

Long-time Jenkins user, but this is the first time I’ve ever attempted anything this complicated and pipelines are a new concept for me.

Rich
  • 117
  • 1
  • 1
  • 10
  • Do [lockable resources](https://jenkins.io/doc/pipeline/steps/lockable-resources/) suit your needs? These are very lightweight and you can filter by label. – zett42 Feb 15 '20 at 10:22
  • @zett42 Yes, that might just do the trick! Thank you I will try the lockable resources plugin and report back with my results. – Rich Feb 15 '20 at 15:36
  • @zett42 I finally got a chance to try lockable resources, and it seems there's a quirk with the `lock(label: 'ethernet')` form used in Declarative Pipeline syntax: you have to supply a `resource` parameter when you supply a `label` parameter, which seems to defeat the purpose of labels. Found an example of locking by label here [link](https://jenkinsci.github.io/job-dsl-plugin/#method/javaposse.jobdsl.dsl.jobs.IvyJob.lockableResources) but that seems to be using the scripted pipeline syntax. Anyone know of a way to use `lock(label: 'xxx')` without supplying a `resource` name? – Rich Feb 24 '20 at 21:29
  • I should have posted the actual error I was getting: `WorkflowScript: 30: Missing required parameter: "resource" @ line 30, column 25. lock(label: 'ethernet') {`. The documentation (link in first comment) explicitly says _Either a resource or a label need to be specified_. – Rich Feb 24 '20 at 21:38
  • I have used it in declarative aswell as scripted pipeline. In either case I only had to specify a label, not a resource, which adheres to the documentation. It might be interesting to know that you can use `lock` as a stage option in declarative pipeline, which is missing from the documentation. I can post some examples as an answer when I'm back at work tomorrow. – zett42 Feb 24 '20 at 21:45
  • BTW, [this answer](https://stackoverflow.com/a/56911603/7571258) suggests that you can specify `null` for the resource argument, although I don't remember having to use that. Maybe that's because of my use of `lock` as a stage option. – zett42 Feb 24 '20 at 21:50
  • Thank you! I located an example of using `lock(..)` as a stage option, and now I don't get any errors. But .. the lock doesn't seem to be doing what I expected. I have two stages (inside a `parallel` block) and each of those stages does a `lock(label: 'ethernet')`. I have two Global resources defined, each with the word `ethernet` in its label string. I have 4 executors, so I am _expecting_ the two stages to execute at the same time but one always waits until the first completes. If I add a third stage (with no `lock(..)` option) it _does_ run in parallel. I'm confused. – Rich Feb 24 '20 at 22:51
  • 1
    Wait, I missed the `quantity` parameter to `lock(..)`. It seems if you omit `quantity` it assumes **all** and that wasn't what I wanted. If I change the lock call to `lock(label: 'ethernet', quantity: 1)` I get the behaviour I was expecting. @zett42 if you want to weigh in with your observations I'll accept your answer and we're done here. – Rich Feb 24 '20 at 22:58

1 Answers1

1

A straightforward way is to use the Lockable Resources plugin.

This can be used as a step as well as a stage option (undocumented). The latter comes in handy if you have nested stages which all depend on the resource to be locked.

Stage option in declarative pipeline

pipeline {
    agent any

    stages {
        stage('Test') {
            options {
                // Lock a single resource from all resources labeled 'mylabel'
                lock( label: 'mylabel', 
                      quantity: 1,
                      variable: 'MyResourceName' )
            }
            steps { // or 'parallel' or 'stages'
                echo "Locked resource $MyResourceName"
                sleep 10
                echo "Resource will be unlocked after this stage"
            }
        }
    }
}

Step in scripted pipeline

node {
    stage('Test') {
        lock( label: 'mylabel', 
              quantity: 1,
              variable: 'MyResourceName' ) {

            echo "Locked resource $MyResourceName"
            sleep 10
            echo "Resource will be unlocked after this stage"
        }
    }
}

Caveats

  • If lock is used as a step in declarative pipeline, you may get an error:

    Missing required parameter: "resource"

    This seems to be a little bug in argument checking. According to the documentation, you only need to specify either resource or label parameter. Simply pass null as the value for this parameter.

  • If parameter quantity is not specified, all resources that match the given label will be locked.

zett42
  • 25,437
  • 3
  • 35
  • 72
  • Nice summary of the solution and the issues that may be encountered getting there. Thanks! – Rich Feb 25 '20 at 13:51