Jenkins JUnit Plugin reports a build as unstable even if test fails

Question

So I am using the Jenknis JUnit parser plugin to parse the test results. I have a job which has just one test and it fails all the time. However, the JUnit plugin marks the job as unstable and not failed.

Any reasons why?

I have tried to set the Health report amplification factor to 1, 0.1, 0.0 but no luck. Seems like somehow this is the reason why my job is reported as Unstable and not Failed.

How can I get the JUnit to fail the build?

Thanks!

Answer 1

The following workaround worked for me:

sh "test ${currentBuild.currentResult} != UNSTABLE"

I added this right after the junit step. So in the end your pipeline looks similar to this:

stage('test') {
    steps {
        sh "mvn -Dmaven.test.failure.ignore=true -DtestFailureIgnore=true test"
    }
    post {
       success {
           junit '**/target/surefire-reports/**/*.xml'
           sh "test ${currentBuild.currentResult} != UNSTABLE"
       }
    }
}

Answer 2

Unfortunately JUnit reporter doesn't support failing the build. This is quite frustrating, because that's kind of a popular use case...

In any case, you need to implement a workaround. So you either:

1. Run the unit tests with allowing the test tool to fail the build (just let it throw it's non-0 exit code) or...

2. Save the exit code(s) of the test tool(s) and fail the build later. You can do that for any type of job (incl. Pipeline). For usual job then the Execute shell step would look smth like this:

   set +e                      # Disable the option to fail the job when any command fails
   phpunit                     # Run the tests
   unit_tests_exit_code=$?     # Save the exit code of the last command
   set -e                      # Re-enable the option to fail the job when any command fails
   
   echo "yolo"                 # Execute everything you want to run before you fail the job
   
   exit $unit_tests_exit_code  # You fail the job (if test run failed and phpunit returned a non-zero exit code)
   

Answer 3

Incase, where the tests are run within a container and one wants to get the status of the tests from exit code on the container, below commands can be used

test_status=$(docker inspect <containerName/ID> --format='{{.State.ExitCode}}')
exit $test_status