Multiple agents in Jenkins pipeline but use one agent for certain stages

Question

I have this Jenkinsfile:

#!groovy
pipeline
{
options {
    buildDiscarder(logRotator(numToKeepStr: '10'))
}

agent {

    label 'docker && new'

}

stages
{

    stage('Docker build')

    {
        when {
            branch 'dev'
        }

        steps
        {                 


            sh "echo ${env.BUILD_NUMBER}"
            sh "./scripts/push.sh Docker http://xxxxx.xxxx ${env.BUILD_NUMBER} ${env.GIT_BRANCH}"
            sh "echo ${env.BUILD_NUMBER}"
            sh "echo ${env.GIT_BRANCH}"

        }
    }

    stage("Initialise") 
    {

        agent {
            dockerfile {
            filename 'Dockerfile'
            label 'docker && new'
            args '--entrypoint ""'
            }
        }

         steps {

                sh "terraform init -input=false"
        }


    }

    stage("WorkspaceDev") {

        agent {
            dockerfile {
            filename 'Dockerfile'
            label 'docker && new'
            args '--entrypoint ""'
            }
        }

        when {
            branch 'dev'
        }

        steps {

                sh "terraform workspace select dev || terraform workspace new dev"
        }

    }

}
}

It builds a container from my Dockerfile, However when running this job it is creating a new docker container to run the next stage called WorkspaceDev. I need to use a separate agent for the very first stage and then dockerfile agent for all other stages

How can I use the same container built for the Initialise stage?

Problem:

When running this pipeline the "Docker build" stage is executing on the agent itself as expected. It then gets to the "initialisation" stage. This build a new docker container (docker build (my Dockerfile I have specified in the agent section for this stage). This stage completes inside this container. Next it gets to the "WorkspaceDev" stage - this then AGAIN rebuilds the container with docker build.

I want to use the same container built in the "Initialisation" stage

Have you tried the `docker` agent instead? Also, I think your individual agents will be ignored unless you set `agent none` at the highest scope. — Matthew Schuchard, Jan 31 '20 at 15:12
@MattSchuchard Individual agents are not ignored. I used the dockerfile agent as I have a custom docker container I need to build for the stage execution — W Khan, Jan 31 '20 at 16:01

AnonymousLama · Answer 1 · 2020-01-31T15:20:22.090

0

Can't you use agent { label 'NODE_LABEL' }?
Your question is not very clear, but it seem you are defining a global agent in the upper part of your Jenkinsfile. If the stage you are trying to run on a different agent is the "Docker build" stage, you just have to specify an agent like in your other stages.

There is the agent documentation: https://jenkins.io/doc/book/pipeline/syntax/#agent

edited Jan 31 '20 at 15:20

answered Jan 31 '20 at 13:31

AnonymousLama

62
10

What do you mean? Sorry very noob at pipelines – W Khan Jan 31 '20 at 13:45
@AnonymousLama Thanks for the answer but how will adding a label help? I have added some more clarification to the question – W Khan Jan 31 '20 at 15:57
I've found this answer about multiple steps on the same container: https://stackoverflow.com/a/57636481/12764126 Basically says to user docker instead of dockerfile. Some tweaks may need to be done for the parameters. You can look at the documentation on my first answer (just search for keyword docker, its a little bit lower on the page) – AnonymousLama Jan 31 '20 at 16:03
@AnonymousLama Great awesome, thank you so much. I did not know you can nest stages, thanks – W Khan Jan 31 '20 at 16:16
1

You're welcome! I'm not really used to this part, but the documentation is well done. Glad I could give you a hint! – AnonymousLama Jan 31 '20 at 16:18

score 0 · Answer 2 · answered Feb 02 '20 at 04:22

I think that the label has no meaning (so it is ignored) when you're using a Dockerfile-agent, so it creates a new container for each stage.

therefore, try the following:

#!groovy
pipeline
{
options {
    buildDiscarder(logRotator(numToKeepStr: '10'))
}

agent {
   dockerfile {
      filename 'Dockerfile'
      args '--entrypoint ""'
   }
}

stages
{

    stage('Docker build')

    {
        agent {
            dockerfile {
            filename 'Dockerfile'
            args '--entrypoint ""'
            }
        }
        when {
            branch 'dev'
        }

        steps
        {                 


            sh "echo ${env.BUILD_NUMBER}"
            sh "./scripts/push.sh Docker http://xxxxx.xxxx ${env.BUILD_NUMBER} ${env.GIT_BRANCH}"
            sh "echo ${env.BUILD_NUMBER}"
            sh "echo ${env.GIT_BRANCH}"

        }
    }

    stage("Initialise") 
    {


         steps {

                sh "terraform init -input=false"
        }


    }

    stage("WorkspaceDev") {

        when {
            branch 'dev'
        }

        steps {

                sh "terraform workspace select dev || terraform workspace new dev"
        }

    }

}
}

Multiple agents in Jenkins pipeline but use one agent for certain stages

2 Answers2