Gradle : How to generate coverage report for Integration test using jacoco

Question

I am new to gradle. I am using the below code. But it generates coverage for unit test cases. But it didn't generate for integration test cases. I have my test classes in the package src/test/java.

test {
    dependsOn jettyRunWar
    ignoreFailures true
    finalizedBy jettyStop
}

apply plugin: 'jacoco'

jacocoTestReport {
    group = "Reporting"
    description = "Generate Jacoco coverage reports after running tests."
    additionalSourceDirs = files(sourceSets.main.allJava.srcDirs)
}

Answer 1

EDIT4: Gradle 7.4 RC1 release notes indicates gradle has now the possibility to generate a single report file for both JUnit and JaCoCo. This will avoid the brittle configuration explained below.

All you have to do is apply the relevant plugins

• test-report-aggregation
• jacoco-report-aggregation

The current downside at this moment (7.4 RC1) is that only the HTML reports are supported. And that these aggregation tasks work in concert with the JVM test suite plugin (yet automatically added by the java plugin).

So keep an eye on this feature in the next release.

Using Gradle 5.4.1 (and now 5.5.1), I was able to get a report after any test task, currently I have both test and integrationTest tasks.

EDIT3: Fixed a potential bug when executing only some test tasks

• Don't configure executionData in doLast / doFirst blocks, it was an error from my part. For more information checks this gradle github ticket
• Added the more prudent alternative (again not in a doLast / doFirst block) executionData { tasks.withType(Test).findAll { it.jacoco.destinationFile.exists() }*.jacoco.destinationFile }

EDIT2: The solution is the same, I just tweaked

• the reports destination to use jacoco.reportsDir,
• the executionData now takes tasks.withType(Test) instead of just [test, integrationTest]
• setting executionData is done in the doFirst block instead of doLast

EDIT: After looking at the documentation of JacocoReport, there's a variant JacocoReport:executionData that take Gradle tasks directly. It works because the JaCoCo plugin adds a JacocoTaskExtension extension to all tasks of type Test. Which is then less error prone.

---

jacocoTestReport {
    // The JaCoCo plugin adds a JacocoTaskExtension extension to all tasks of type Test.
    // Use task state to include or not task execution data
    // https://docs.gradle.org/current/javadoc/org/gradle/api/tasks/TaskState.html
    // This declaration will be used as a closure, notice there no wrapping parenthesis
    executionData tasks.withType(Test).findAll { it.state.executed }
    
    // If the above instruction really don't work, there maybe some things that intervene in the process, in this case, you may be a bit more lucky with this instruction
    // executionData { tasks.withType(Test).findAll { it.jacoco.destinationFile.exists() }*.jacoco.destinationFile }

    reports {
        xml.enabled true
        xml.destination(file("${jacoco.reportsDir}/all-tests/jacocoAllTestReport.xml"))
        html.enabled true
        html.destination(file("${jacoco.reportsDir}/all-tests/html"))
    }
}

And the same trick can be applied to sonarqube task :

sonarqube {
    group = "verification"
    properties {
        // https://jira.sonarsource.com/browse/MMF-1651
        property "sonar.coverage.jacoco.xmlReportPaths", jacocoTestReport.reports.xml.destination
        properties["sonar.junit.reportPaths"] += integrationTest.reports.junitXml.destination
        properties["sonar.tests"] += sourceSets.integrationTest.allSource.srcDirs
        // ... other properties
    }
}

---

Older but very working answer. Also using the knowledge above (that Tests task are extended by JacocoTaskExtension) it's possible to replace the manual file configuration of executionData by test.jacoco.destinationFile and integrationTest.jacoco.destinationFile.

// Without it, the only data is the binary data, 
// but I need the XML and HTML report after any test task
tasks.withType(Test) {
    finalizedBy jacocoTestReport
}

// Configure the report to look for executionData generated during the test and integrationTest task
jacocoTestReport {
    executionData(file("${project.buildDir}/jacoco/test.exec"),
                  file("${project.buildDir}/jacoco/integrationTest.exec"))
    reports {
        // for sonarqube
        xml.enabled true
        xml.destination(file("${project.buildDir}/reports/jacoco/all-tests/jacocoAllTestReport.xml"))
        // for devs
        html.enabled true
        html.destination file("${project.buildDir}/reports/jacoco/all-tests/html")
    }
}


sonarqube {
    group = "verification"
    properties {
        // https://jira.sonarsource.com/browse/MMF-1651
        property "sonar.coverage.jacoco.xmlReportPaths", ${project.buildDir}/test-results/integrationTest"
        properties["sonar.junit.reportPaths"] += "${project.buildDir}/test-results/integrationTest"
        properties["sonar.tests"] += sourceSets.integrationTest.allSource.srcDirs
        // ... other properties
    }
}

project.tasks["sonarqube"].dependsOn "jacocoTestReport"

Answer 2

I believe the most full answer will look like:

tasks.withType(Test) {
    finalizedBy jacocoTestReport
}

project.jacocoTestReport {
    getExecutionData().setFrom(fileTree(buildDir).include("/jacoco/*.exec"))

    reports {
        csv.enabled true
    }
}

At least it's fully suited for my needs with integration and functional testing.

Answer 3

Since I couldn't make it run with any of the answers, I will add my solution here. It will work if you run your test task (e.g. integTest) first and call the jacocoTestReport afterwards.

All you need is to tell the jacocoTestReport task where to find the gathered execution data from you test task. The execution data is always named after the test task. So if you have a test task called integTest, your execution data will be stored in build/jacoco/integTest.exec. The jacocoTestReport task can be configured to look for those other files too by adding them to the property executionData. You can also add wildcards includes so all execution data is taken into consideration:

jacocoTestReport {
    executionData = fileTree(dir: project.projectDir, includes: ["**/*.exec"])
}

UPDATE As stated by @rahulmohan the executionData property has become readonly. Instead define the jacocoTestReport task as below:

jacocoTestReport {
    getExecutionData().from(fileTree(project.projectDir).include("/jacoco/*.exec"))

by executing the statement below the test coverage jacoco report will be created for you integration test task (e.g. integTest)

./gradlew integTest jacocoTestReport

This also works for multi module projects where you want to run the integTest task in module a:

./gradlew a:integTest a:jacocoTestReport

Answer 4

It seems like, what you need to tell build.gradle is where are your Intergration tests (i.e. folder containing those IT tests) using sourceSets. In my case, i have source under src/java (instead of src/main/java - gradle default).. my unit tests (Junit) under test/java folder, and my integration tests under src/java-test folder.

sourceSets {
   main {
      java {
         srcDir 'src/java'
      }
   }
   test {
      java {
         srcDir 'test/java'
      }
      resources {
         srcDir 'test/resources'
         srcDir 'conf'
      }
   }
   integrationTest {
      java {
         srcDir 'src/java-test'
      }
   }
}

Then, I have integrationTest task as ... which you can tweak as you might not have cleanTest (custom task that i have created), so you can ignore that dependsOn... i think in your case you'll use something like jettyStart as you're using that for IT tests (starting the container for running IT tests and then finalizedBy feature to stop jetty .. jetty plugin)

task integrationTest( type: Test, dependsOn: cleanTest ) {
   jacoco {
      //destinationFile = file("$buildDir/jacoco/jacocoTest.exec")
      destinationFile = file("$buildDir/jacoco/integrationTest.exec")
      //classDumpFile = file("$buildDir/jacoco/classpathdumps")
      classDumpFile = file("$buildDir/classes/integrationTest")
   }
   testClassesDir = sourceSets.integrationTest.output.classesDir
   classpath = sourceSets.integrationTest.runtimeClasspath
}

SEE this post for more detailed output structure and script that I have at my end. Im getting the .exec for both Unit tests (test.exec) and IT tests intergrationTest.exec.. but Im not getting the jacoco.xml/jacocoHtml reports for both tests. I also found that, if I run "gradle clean build" (which includes call to "test" task) and "gradle clean build integrationTest" then, later one overwrites unit tests data in build/test-results folder and build/reports/tests folder.

Jacoco Unit and Integration Tests coverage - individual and overall

NOTE: in my case, jacocoTestReport is defined in the global gradle init.d folder in one of the common gradle file. This will help us not to include the same code in all / at project level build.gradle file.