Jenkins pipelines with parallel and different containers

Question

So I am already running Jenkins pipelines with parallel base on the example from: Is it possible to create parallel Jenkins Declarative Pipeline stages in a loop?

I want to run each job in different isolated container, the agent name should be the same to all of them. Tried a few options all of them ended up withe errors, I think I need to use both declarative and scripted but not sure how. Things I tired:

def generateTerraformStage(env) {
    return {
        agent { label 'local_terraform' }
        stage("stage: Terraform ${TERRAFORM_ACTION} ${env}") {
                echo "${env}"
                sleep 30
        }
    }
} 

stage('parallel stages') {
    agent { label 'local_terraform' }
    steps {
        script {
            parallel parallelStagesMapEnvironment
        }
    }
}

One of the errors I got during testing:

"java.lang.NoSuchMethodError: No such DSL method 'agent' found among steps" and "java.lang.IllegalArgumentException: Expected named arguments but got org.jenkinsci.plugins.workflow.cps.CpsClosure2@560f3533"

Answer 1

Dynamic parallel stages could be created only by using Scripted Pipelines. The API built-it Declarative Pipeline is not available (like agent, options, when etc.).

I don't see any information that you really need dynamic stages (e.g. based on the value returned by a 3rd-party service), so I prepared two solutions:

• dynamic parallel stages - stages are generated based on something
• static parallel stages - you know all stages (the when block could be used to disable these which are not needed - e.g. passed in parameters)

pipeline {
  // ...
  stages {
    stage('dynamic parallel stages') {
      steps {
        script {
           // params.ENVS == ['envA', 'envB', 'envC']
           def values = params.ENVS.split(',')
           def stages = [:]
           for (def value in values) {
             stages[value] = generateTerraformStage(value)
           }
           parallel stages
        }
      }
    }
    stage('static parallel stages') {
      parallel {
        stage('envA') {
          agent { label 'local_terraform' }
          when {
            expression { return params.ENVS.split(',').contains('envA') }
          }
          steps {
            terraformStageLogic 'envA'
          }
        }
        stage('envB') {
          agent { label 'local_terraform' }
          when {
            expression { return params.ENVS.split(',').contains('envB') }
          }
          steps {
            terraformStageLogic 'envB'
          }
        }
        stage('envC') {
          agent { label 'local_terraform' }
          when {
            expression { return params.ENVS.split(',').contains('envC') }
          }
          steps {
            terraformStageLogic 'envC'
          }
        }
        // ...
      }
    }
  }
}

Closure<Void> generateTerraformStage(env) {
  return {
    node('local_terraform') {
      stage("stage: Terraform ${TERRAFORM_ACTION} ${env}") {
        echo "${env}"
        sleep 30
      }
    }
  }
}

void terraformStageLogic(env) {
  echo "${env}"
  sleep 30
}

When you don't use the workspace in the stage responsible for generating or executing other stages (dynamic parallel stages and static parallel stages) then you don't need to allocate any node to it (waste of resources).