How to write a dynamic declarative pipeline that contains sequential job inside parallel job

Question

I'm trying to write a declarative pipeline code that accepts a map and create a pipeline. I can able to achieve sequential stages or parallel stages but facing problems while making a pipeline that contains sequential stages inside parallel stages.

The input data would be Map. Each list in the map should run parallel and the items inside the list corresponding to each key should run in sequentially.

example data : [1:[11,12], 2:[21,22], 3:[31,32]]

The output should be of image. Could someone give some idea?

Below is the code i have tried.

def stageData = [1:[11,12], 2:[21,22], 3:[31,32]];

def getDeployStages1(stageData){
    Map deployStages = [:]
    stageData.each{ key, stgValue ->
        List stgs = []
        stgValue.each{ value ->           
            deployStages.put("${value}", {
                echo "${value}"
            })
        }        
    }
    return deployStages;
}

def getDeployStages2(stageData){
    Map deployStages = [:]
    stageData.each{ key, stgValue ->
        List stgs = []
        stgValue.each{ value ->
            stgs.add(stage("${value}"){
               echo "${value}"
            })           
        }
        deployStages.put("${key}", stgs)
    }
    return deployStages;
}

pipeline {
    agent any
    stages {
        stage ("deploy1") {
            steps {
                script {
                     parallel getDeployStages1(stageData)
                }
            }
        }
        stage ("deploy2") {
            steps {
                script {
                    parallel getDeployStages2(stageData)
                }
            }
        }
    }
}

I have added the code. I tried the declarative pipeline. But am ok to have the solution with the scripted pipeline. — june, May 20 '19 at 08:46

score 0 · Answer 1 · answered May 20 '19 at 09:02

According to this documentation you can nest the stages in this way

pipeline {
    agent none

    stages {
        stage("build and deploy on Windows and Linux") {
            parallel {
                stage("windows") {
                    agent {
                        label "windows"
                    }
                    stages {
                        stage("build") {
                            steps {
                                bat "run-build.bat"
                            }
                        }
                        stage("deploy") {
                            when {
                                branch "master"
                            }
                            steps {
                                bat "run-deploy.bat"
                            }
                        }
                    }
                }

                stage("linux") {
                    agent {
                        label "linux"
                    }
                    stages {
                        stage("build") {
                            steps {
                                sh "./run-build.sh"
                            }
                        }
                        stage("deploy") {
                             when {
                                 branch "master"
                             }
                             steps {
                                sh "./run-deploy.sh"
                            }
                        }
                    }
                }
            }
        }
    }
}

This should result in the following flow

To apply this in your case, you can simplify your functions to return just elements that need to be sequential (just the values).

pipeline {
    agent any
    stages {
        stage ("parallel") {
        parallel {
        stage ("deploy1") {
            stages {
                 def list = getDeployStages1(stageData)
                 for (int i=0; i < list.size(); i++) {
                      stage(i) {
                          echo("${list[i]}")
                      }
            }
        }
        stage ("deploy2") {
            stages {
                //similar
            }
        }
        }
    }
}

Yes, I have read the documentation before trying out the thing. But it is difficult to implement as dynamic or at this point impossible. The iteration should go inside the "script". Inside the "script", either we can provide the stage directly to run sequentially or stages in parallel. The parallel accepts only Map. — june, May 20 '19 at 10:18
Why? Where does this requirement come from? And more interestingly, did you try this approach of mine? — hakamairi, May 20 '19 at 12:25
yes. The loop should come inside the script. Assume you need to deploy in an environment and it has multiple servers in multiple hosts. the deployment can be started in parallel in each host but sequentially inside the single host. [host1: [server1,server2], host2: [server1, server2]] — june, May 21 '19 at 10:38
it seems not allowed. java.lang.NoSuchMethodError: No such DSL method 'stages' found among steps — june, May 21 '19 at 12:58

How to write a dynamic declarative pipeline that contains sequential job inside parallel job

1 Answers1