Jenkins:How to Achieve parallel dynamic stages in jenkins declarative pipeline

Question

I am working on declarative pipeline. I am trying to achieve dynamic stages which should distribute the stages parallel with the agents defined. When i explored i learned how to achieve Dynamic sequential stages. Below is my sample code.

My problem now is, how to achieve parallel stages with agents i have. For example, if i have 3 agents all the 5 stages should run parallel in the agents parallel. I tried using parallel tests but not working. Please help me to improve further !

def learn
pipeline {
    agent none

    stages {
        stage('Dynamic Stages') {
          steps {
                script {
                    learn = ["1", "2", "3", "4", "5"]
                    for(int i=0; i < list.size(); i++) {

                        stage(list[i]){
                            echo "value: $i"
                        }
                    }
                }
            }

        }
    }
}

It is not very clear what you are trying to obtain, can you try to be more specific? What behaviour do you get, what behaviour would you expect? — Chris Maes, Dec 10 '19 at 07:59
My exact request is: I wanted to achieve a parallel stages which will have to run stages based on the Agent available list. For example if i have 3 agents online, my test want to distribute in available 3 agents. Please let me know if u still need more details. Thanks — Kaushik, Dec 10 '19 at 08:03
This definitely is a dupe of this SO post: https://stackoverflow.com/questions/46894308/is-it-possible-to-create-parallel-jenkins-declarative-pipeline-stages-in-a-loop That link has a lot more detail and info. — Max Cascone, Jan 13 '22 at 16:14

score 1 · Accepted Answer · answered Dec 10 '19 at 08:24

1

The following should run all stages in parallel. Jenkins will just take whatever node is available.

def learn
pipeline {
    agent none

    stages {
        stage('Dynamic Stages') {
          steps {
                script {
                    learn = ["1", "2", "3", "4", "5"]
                    def builders = [:]
                    for(i in learn) {
                        def value = i // Need to bind the label variable before the closure - can't do 'for (i in learn)
                        builders[i] = {
                            node {
                                echo "value: $i"
                            }
                        }
                    }
                    parallel builders
                }
            }
        }
    }
}

answered Dec 10 '19 at 08:24

Chris Maes

35,025
12
111
136

Hey Chris, Thanks for the suggestion code snippet. I have a clarification here. I believe Builders array takes the node details and parallel builders runs the parallel tests. Here where to define the Agent details. For example Builder[0] should run in Node/Agent "Machine 1" – Kaushik Dec 10 '19 at 08:42
in that case you can specify the label for your node like this: `node(label)` – Chris Maes Dec 10 '19 at 08:43
1

cool enough !, i got what i wanted to achieve. will mark down as answered. May i know is there any documentation from jenkins side for this scenario – Kaushik Dec 10 '19 at 08:46
I don't remember where I found this, I patched it together from different sources... we are using a similar construct here... – Chris Maes Dec 10 '19 at 08:47

Jenkins:How to Achieve parallel dynamic stages in jenkins declarative pipeline

1 Answers1

Linked