Jenkins Workflow: Parallelize step based on tool output

Question

I want to parallelize Jenkins stages based on the output of a test tool. However, I am encountering a problem because all parallel nodes are getting defined equally (in addition to the currently broken loops in the jenkins-workflow plugins). Trimmed down workflow script example:

instances = ["one", "two", "three"]
print "Testing instances: " + instances

test_nodes = [:]
for (int i = 0; i < instances.size(); i++) {
  instance_name = instances.get(i)
  println "Processing instance " + instance_name
  test_nodes["tk-${instance_name}"] = {
    node {
      stage name: ('stage ' + instance_name)
      echo instance_name
    }
  }
}
echo "test_nodes: ${test_nodes}"
parallel test_nodes

While I would expect a result as follows:

node {
  stage name: 'stage one'
  echo 'one'
},
node {
  stage name: 'stage two'
  echo 'two'
},
node {
  stage name: 'stage three'
  echo 'three'
}

I get all three nodes defined as three - as it can be seen in the following output (notice the repeated output three):

[Pipeline] echo
Testing instances: [one, two, three]
[Pipeline] echo
Processing instance one
[Pipeline] echo
Processing instance two
[Pipeline] echo
Processing instance three
[Pipeline] echo
test_nodes: [tk-one:org.jenkinsci.plugins.workflow.cps.CpsClosure2@3febb2f8, tk-two:org.jenkinsci.plugins.workflow.cps.CpsClosure2@b32d891, tk-three:org.jenkinsci.plugins.workflow.cps.CpsClosure2@37281d55]
[Pipeline] Execute in parallel : Start
[Pipeline] [tk-one] parallel { (Branch: tk-one)
[Pipeline] [tk-two] parallel { (Branch: tk-two)
[Pipeline] [tk-three] parallel { (Branch: tk-three)
[Pipeline] [tk-one] Allocate node : Start
[tk-one] Running on master in /var/lib/jenkins/jobs/cookbook-pipeline-cookbook-site-mstypo3org/workspace
[Pipeline] [tk-two] Allocate node : Start
[tk-two] Running on master in /var/lib/jenkins/jobs/cookbook-pipeline-cookbook-site-mstypo3org/workspace@2
[Pipeline] [tk-three] Allocate node : Start
[tk-three] Running on master in /var/lib/jenkins/jobs/cookbook-pipeline-cookbook-site-mstypo3org/workspace@3
[Pipeline] [tk-one] node {
[Pipeline] [tk-two] node {
[Pipeline] [tk-three] node {
[Pipeline] [tk-one] echo
[tk-one] three
[Pipeline] } //node
[Pipeline] [tk-two] echo
[tk-two] three
[Pipeline] } //node
[Pipeline] [tk-three] echo
[tk-three] three
[Pipeline] } //node
[Pipeline] Allocate node : End
[Pipeline] Allocate node : End
[Pipeline] Allocate node : End
[Pipeline] } //parallel
[Pipeline] } //parallel
[Pipeline] } //parallel
[Pipeline] Execute in parallel : End
[Pipeline] End of Pipeline

Why is this happening? Is this another bug in workflow-cps, or am I getting something wrong? The parallel example does not access any variables.

score 9 · Accepted Answer · edited Jun 29 '17 at 19:15

Try def instance_name = instances.get(i)

Without it, its like you just have a references to the instance_name, which ends up as three after the loop. Its the same object that way..

Result:

[Pipeline] echo
Testing instances: [one, two, three]
[Pipeline] echo
Processing instance one
[Pipeline] echo
Processing instance two
[Pipeline] echo
Processing instance three
[Pipeline] echo
test_nodes: [tk-one:org.jenkinsci.plugins.workflow.cps.CpsClosure2@4cb28325, tk-two:org.jenkinsci.plugins.workflow.cps.CpsClosure2@5bc01979, tk-three:org.jenkinsci.plugins.workflow.cps.CpsClosure2@20c885fe]
[Pipeline] Execute in parallel : Start
[Pipeline] [tk-one] parallel { (Branch: tk-one)
[Pipeline] [tk-two] parallel { (Branch: tk-two)
[Pipeline] [tk-three] parallel { (Branch: tk-three)
[Pipeline] [tk-one] Allocate node : Start
[tk-one] Running on master in /var/lib/jenkins/jobs/pipeline/workspace
[Pipeline] [tk-two] Allocate node : Start
[tk-two] Running on master in /var/lib/jenkins/jobs/pipeline/workspace@2
[Pipeline] [tk-three] Allocate node : Start
[Pipeline] [tk-one] node {
[Pipeline] [tk-two] node {
[tk-three] Running on master in /var/lib/jenkins/jobs/pipeline/workspace
[Pipeline] [tk-one] stage (stage one)
[tk-one] Entering stage stage one
[tk-one] Proceeding
[Pipeline] [tk-one] echo
[tk-one] one
[Pipeline] } //node
[Pipeline] [tk-two] stage (stage two)
[tk-two] Entering stage stage two
[tk-two] Proceeding
[Pipeline] [tk-two] echo
[tk-two] two
[Pipeline] } //node
[Pipeline] Allocate node : End
[Pipeline] [tk-three] node {
[Pipeline] Allocate node : End
[Pipeline] } //parallel
[Pipeline] } //parallel
[Pipeline] [tk-three] stage (stage three)
[tk-three] Entering stage stage three
[tk-three] Proceeding
[Pipeline] [tk-three] echo
[tk-three] three
[Pipeline] } //node
[Pipeline] Allocate node : End
[Pipeline] } //parallel
[Pipeline] Execute in parallel : End
[Pipeline] End of Pipeline
Finished: SUCCESS

Read up on variable scope and binding mechanics i.e. here and here.

Thanks for helping me as a Groovy novice! Will read through the cited docs. — StephenKing, Apr 15 '16 at 09:44
Welcome, also [this doc](https://jamesgdriscoll.wordpress.com/2012/04/14/groovy-bindings-or-adding-keywords-to-your-dsl/) about groovy binding mechanics is helpful. — Dominik Gebhart, Apr 15 '16 at 10:36

Jenkins Workflow: Parallelize step based on tool output

1 Answers1

Linked