How to get output of jenkins pipeline in a specific format?

Question

I am trying to implement Machine learning in my jenkins pipeline.
For that I need output data of pipeline for each build.

Some parameters that i need are:

Which user triggered the pipeline
Duration of pipeline
Build number with its details
Pipeline pass/fail
If fail, at which stage it failed.
Error in the failed stage. (Why it failed)
Time required to execute each stage
Specific output of each stage (For. eg. : If a stage contains sonarcube execution then output be kind of percentage of codesmells or code coverage)

I need to fetch these details for all builds. How can get it?

There is jenkins api that can be implemented in python but i was able to get only JOB_NAME, Description of job, IS job Enabled. These details werent useful.

Raoslaw Szamszur · Accepted Answer · 2018-10-27T19:14:28.490

There are 2 ways to get some of data from your list.

1. Jenkins API

For first 4 points from the list, you can use JSON REST API for a specific build to get those data. Example API endpoint:

https://[JENKINS_HOST]/job/[JOB_NAME]/[BUILD_NUMBER]/api/json?pretty=true

1. Which user triggered the pipeline

This will be under actions array in response, identyfi object in array by "_class": "hudson.model.CauseAction" and in it you will have shortDescription key which will have that information:

"actions": [
        {
            "_class": "hudson.model.CauseAction",
            "causes": [
                {
                    "_class": "hudson.triggers.SCMTrigger$SCMTriggerCause",
                    "shortDescription": "Started by an SCM change"
                }
            ]
        },

2. Duration of pipeline

It can be found under key: "duration". Example

"duration": 244736,

3. Build number with its details

I don't know what details you need, but for build number look for "number" key:

"number": 107,

4. Pipeline pass/fail

"result": "SUCCESS",

If you need to extract this information for all builds, run GET request for job API https://[JENKINS_HOST]/job/[JOB_NAME]/api/json?pretty=trueand extract all builds, then run above-mentioned request per build you have extracted.

I will write later a dummy python script to do just that.

2. Dump data in Jenkinsfile

There is also a possibility to dump some that information from Jenkinfile in post action.

pipeline {
agent any

stages {
    stage('stage 1') {
        steps {
            sh 'echo "Stage 1 time: ${YOUR_TIME_VAR}" > job_data.txt'
        }
    }
}
post {
    always {
        sh 'echo "Result: ${result}" > job_data.txt'
        sh 'echo "Job name: ${displayName}" > job_data.txt'
        sh 'echo "Build number: ${number}" > job_data.txt'
        sh 'echo "Duration: ${duration}" > job_data.txt'


        archiveArtifacts artifacts: 'job_data.txt', onlyIfSuccessful: false
    }
}

}

List of available global variables for pipeline job can be found:

https://[JENKINS_HOST]/pipeline-syntax/globals#env

For rest, you will need to implement your own logic in Jenkinsfile.

Ad. 5

Create a variable which holds information about current stage. At the beginning of each stage change its value to the ongoing stage. At the end dump to file like rest variables. If pipeline will fail let's say on stage foo in post action this variable will have exact same value because if pipeline fails it won't go to next stage.

Ad. 6 I'm not sure what you want, a traceback, error code? I guess you will probably need to implement your own logging function.

Ad. 7 Make a function for measuring time for each stage and dump value at the end.

Ad. 8 Also not sure what you mean. Like, build artifacts?

At the end of each build this file job_data.txt will be archived as build artifact which can be later downloaded.

If i will find more elegant and simple solution I'll edit this post.

Hope it helps in any way

EDIT 1

Here is the script I've mentioned earlier.

import requests


username = "USERNAME"
password = "PASSWORD"
jenkins_host = "JENKINS_HOST"
jenkins_job = "JOBNAME"
request_url = "{0:s}/job/{1:s}/api/json".format(
    jenkins_host,
    jenkins_job,
)

job_data = requests.get(request_url, auth=(username, password)).json()

builds = []

for build in job_data.get('builds'):
    builds.append(build.get('number'))

for build in builds:
    build_url = "{0:s}/job/{1:s}/{2:d}/api/json".format(
        jenkins_host,
        jenkins_job,
        build,
    )

    build_data = requests.get(build_url, auth=(username, password)).json()
    build_name = build_data.get('fullDisplayName')
    build_number = build_data.get('number')
    build_status = build_data.get('result')
    build_duration = build_data.get('duration')
    for action in build_data.get('actions'):
        if action.get("_class") == "hudson.model.CauseAction":
            build_trigger = action.get('causes')
    print(build_name)
    print(build_status)
    print(build_duration)
    print(build_number)
    print(build_trigger)

Please note you might need to authorize with API Token depending on your security settings.

How to get output of jenkins pipeline in a specific format?

1 Answers1

1. Jenkins API

2. Dump data in Jenkinsfile

EDIT 1