How can I better visualize the work of a large build step using a Jenkinsfile?

Question

I have a pretty long build step (it takes hours) and a Jenkinsfile triggering it. (The build steps consists of multiple Makefile targets etc.)

Since the build step does everything needed to produce the artifact, the Jenkins Pipeline visualization is pretty useless.

Is there a way how I can better visualize the internals of the build step without porting/duplicating it to a Jenkinsfile ?

Answer 1

You can move away from a single target task to build everything (make or make all) and instead specifically target different build tasks in the Jenkinsfile. For example, let's say you're building an app for different platforms. Your original Jenkinsfile might be...

node('builder') {
  stage('build') {
    sh "make"
  }
}

which was triggering the "all" target in your makefile, which might be triggering the tasks "iOS", "android", and "windowsphone"

Instead, have your Jenkinsfile target each one of these make tasks individually.

Ex:

node('builder') {
  stage('build iOS') {
    sh "make iOS"
  }
  stage('build android') {
    sh "make android"
  }
  stage('build windows phone') {
    sh "make windowsphone"
  }
}

By splitting up your target tasks into these multiple stages, you should better reporting in your Jenkins UI.

If you wanted to be really brave, you could avoid hardcoding this in two places (the all target, and the Jenkinsfile) by creating a script to take apart what all was doing and turn it into jenkins stages. Assuming that your all or default task is just a list of other tasks, you can use the following bash:

if [[ -z $(make -rpn | grep ".DEFAULT_GOAL") ]]; then
    DEFAULTTARGET=all
else
    DEFAULTTARGET=$(make -rpn | grep ".DEFAULT_GOAL" | sed "s/.*=[[:space:]]*//" | tr -d "[:space:]")
fi

make -rpn | sed -n -e "/^$/ { n ; /^[^ ]*:/p ; }" | grep "$DEFAULTTARGET:" | sed "s/.*:[[:space:]]*//" | tr " " "\n"

in a Jenkinsfile like so:

node() {
    ## NOT NORMALLY REQUIRED. Just sets up a simple makefile
    sh "echo 'all: images release clean report' > Makefile "
    ## Start of required logic
    def makeTasks = sh(script: 'if [[ -z $(make -rpn | grep ".DEFAULT_GOAL") ]]; then     DEFAULTTARGET=all; else     DEFAULTTARGET=$(make -rpn | grep ".DEFAULT_GOAL" | sed "s/.*=[[:space:]]*//" | tr -d "[:space:]"); fi; make -rpn | sed -n -e "/^$/ { n ; /^[^ ]*:/p ; }" | grep "$DEFAULTTARGET:" | sed "s/.*:[[:space:]]*//" | tr " " "\n"', returnStdout: true)
    def tasks = makeTasks.split('\n')
    for(int i = 0; i < tasks.size(); i++ ){
        stage(tasks[i]) {
            echo tasks[i]
            // This will fail with the bogus makefile defined above, but would work with a real one.
            // sh 'make ${tasks[i]}'
        }
    }
}

Answer 2

Not really: there was a similar issue for long-running stages.

Regarding steps, there is JENKINS 33185 ("Visualize parallel steps within a Pipeline Stage") still open

But for one single long-running step, all you have is the shell output, which is not that helpful when a command takes a long time to complete.