aggregating gradle multiproject test results using TestReport

Question

I have a project structure that looks like the below. I want to use the TestReport functionality in Gradle to aggregate all the test results to a single directory. Then I can access all the test results through a single index.html file for ALL subprojects. How can I accomplish this?

.
|--ProjectA
  |--src/test/...
  |--build
    |--reports
      |--tests
        |--index.html (testresults)
        |--..
        |--..
|--ProjectB
    |--src/test/...
      |--build
        |--reports
          |--tests
            |--index.html (testresults)
            |--..
            |--..

score 37 · Accepted Answer · edited Mar 30 '20 at 21:12

37

From Example 4. Creating a unit test report for subprojects in the Gradle User Guide:

subprojects {
    apply plugin: 'java'

    // Disable the test report for the individual test task
    test {
        reports.html.enabled = false
    }
}

task testReport(type: TestReport) {
    destinationDir = file("$buildDir/reports/allTests")
    // Include the results from the `test` task in all subprojects
    reportOn subprojects*.test
}

Fully working sample is available from samples/testing/testReport in the full Gradle distribution.

edited Mar 30 '20 at 21:12

togise

278
3
9

answered Jun 04 '13 at 15:29

Peter Niederwieser

121,412
21
324
259

Hey Pete, I need some direction. In my case, I have only one project i.e. ProjectA and after gradle clean build integrationTest, I see build/jacoco/test.exec and build/jacoco/integrationTest.exec. Under build/reports/tests/xxxxxx I see either index.html for Unit tests or only Integration tests i.e. if I run gradle clean build, I see unit tests index.html and if I run gradle clean build integrationTest, then it overwrites the data in build/reports/tests/xxx and gets the new data for integrationTest task in the same folder (build/reports/tests). – AKS Sep 17 '13 at 17:50
Similarly, When I run sonar-runner, I do see both .exec files getting picked during JacocoSensor and I also see jacoco overall .exec getting created in the workspace under .sonar folder and under build/reports/jacoco folder. sonar-runner completes successfully, but I dont see the results in project's dashboard in Sonar even though I have both widgets set for showing Unit tests/coverage and Integration tests. Please advise what I may be missing. – AKS Sep 17 '13 at 17:52
You need to set a separate report dir for integration tests (see [Gradle Build Language Reference](http://gradle.org/docs/current/dsl/index.html)). Not sure about the Sonar problem. – Peter Niederwieser Sep 18 '13 at 04:18
`testReport` is deprecated in Gradle 1.9 and shouldn't be used any more. – ashes999 Nov 23 '13 at 04:52
3

It's only the `test.testReport` property that's deprecated. As the deprecation message says, you just need to replace it with `test.reports.html.enabled = false`. – Peter Niederwieser Nov 23 '13 at 06:26
I'm just saying :) also, this doesn't work unless (like the questioner asked) both projects are in subdirectories; I had my parent project in the root directory, to no avail. – ashes999 Nov 24 '13 at 04:17
The question is about aggregating the test reports of all subprojects. The answer works regardless of the directory layout. – Peter Niederwieser Nov 24 '13 at 06:30
8

I have found that if one test fail, task report is not completed - neither the rest of the tests are executed, so is needed to add `ignoreFailures` to `test` task ([source](https://discuss.gradle.org/t/gradle-if-junits-fail-the-build-is-failing-without-executing-the-next-test-case/6868/7)). – lucasvc Jul 26 '16 at 11:53
How I like when people pointing to /current/ url in documentation, as if this chapter be there forever. – m1ld Oct 10 '19 at 13:50
https://docs.gradle.org/5.6.2/userguide/java_testing.html#test_reporting – m1ld Oct 10 '19 at 13:54
Could not get unknown property 'test' for project ':my-subproject' of type org.gradle.api.Project. – spyro Apr 07 '20 at 08:06
1

It's better to use `reportOn getTasksByName("test", true)` since that'll work if some subprojects doesn't have any test tasks – Raniz May 26 '20 at 06:26

score 4 · Answer 2 · edited Sep 10 '21 at 14:11

4

In addition to the subprojects block and testReport task suggested by @peter-niederwieser above, I would add another line to the build below those:

tasks('test').finalizedBy(testReport)

That way if you run gradle test (or even gradle build), the testReport task will run after the subproject tests complete. Note that you have to use tasks('test') rather than just test.finalizedBy(...) because the test task doesn't exist in the root project.

edited Sep 10 '21 at 14:11

jannis

4,843
1
23
53

answered Sep 11 '19 at 22:52

kousen

2,897
3
28
26

3

I had to do the following fopr Gradle 5.6.2: ```rootProject.getTasksByName('test', true).each { it.finalizedBy(testReport) } ``` – agabrys Nov 21 '19 at 14:19

RenatoIvancic · Answer 3 · 2022-08-23T11:31:58.233

I am posting updated answer on this topic. I am using Gradle 7.5.1.

TestReport task

In short I'm using following script to set up test aggregation form subprojects (based on @Peter's answer):

subprojects {
  apply plugin: 'java'
}

task testReport(type: TestReport) {
  destinationDir = file("$buildDir/reports/allTests")
  // Include the results from the `test` task in all subprojects
  testResults.from = subprojects*.test
}

Note that reportOn method is "deprecated" or will be soon and replaced with testResults, while at the same time testResults is still incubating as of 7.5.1. I got following warning in IDE

The TestReport.reportOn(Object...) method has been deprecated. This is scheduled to be removed in Gradle 8.0.

Hint: subproject*.test is example of star dot notation in groovy that invokes test task on a list of subprojects. Equally would be invocation of subprojects.collect{it.test}

test-report-aggregation plugin

There is also alternative option for aggregating tests (Since Gradle 7.4). One can apply test-report-aggregation plugin. If your projects already apply java plugin, this means they will come with jvm-test-suite, all you have to do is apply the plugin.

plugins {
  id 'test-report-aggregation'
}

Then you will be able to invoke test reports through testSuiteAggregateTestReport task. Personally didn't use the plugin, but I think it makes sense to use it if you have multiple test suites configured with jvm-test-suite.

Example project can be found in https://github.com/gradle-samples/Aggregating-test-results-using-a-standalone-utility-project-Groovy

Juan Rada · Answer 4 · 2020-11-24T16:25:18.133

If using kotlin Gradle DSL

val testReport = tasks.register<TestReport>("testReport") {
    destinationDir = file("$buildDir/reports/tests/test")
    reportOn(subprojects.map { it.tasks.findByPath("test") })

subprojects {
    tasks.withType<Test> {
        useJUnitPlatform()
        finalizedBy(testReport)
        ignoreFailures = true
        testLogging {
            events("passed", "skipped", "failed")
        }
    }
}

And execute gradle testReport. Source How to generate an aggregated test report for all Gradle subprojects

score 0 · Answer 5 · edited Jun 20 '20 at 09:12

0

For 'connectedAndroidTest's there is a approach published by google.(https://developer.android.com/studio/test/command-line.html#RunTestsDevice (Multi-module reports section))

Add the 'android-reporting' Plugin to your projects build.gradle.

apply plugin: 'android-reporting'
Execute the android tests with additional 'mergeAndroidReports' argument. It will merge all test results of the project modules into one report.

./gradlew connectedAndroidTest mergeAndroidReports

edited Jun 20 '20 at 09:12

Community

1
1

answered Apr 12 '17 at 21:15

Lunero

666
1
8
17

Although the doc says it can also merge the unit tests, running this task depends on espresso tests which is not something I was hoping for. – tasomaniac Aug 04 '17 at 10:36

score 0 · Answer 6 · answered Mar 08 '20 at 19:58

0

FYI, I've solved this problem using the following subprojects config in my root project build.gradle file. This way no extra tasks are needed.

Note: this places each module's output in its own reports/<module_name> folder, so subproject builds don't overwrite each other's results.

subprojects {
 // Combine all build results
  java {
    reporting.baseDir = "${rootProject.buildDir.path}/reports/${project.name}"
  }
}

For a default Gradle project, this would result in a folder structure like

build/reports/module_a/tests/test/index.html
build/reports/module_b/tests/test/index.html
build/reports/module_c/tests/test/index.html

answered Mar 08 '20 at 19:58

Joseph Lust

19,340
7
85
83

1

I dont see how this really answer the question. The question asks about ONE index.html with accumulated results. – GhostCat Feb 25 '21 at 12:16
I wanted something exactly like that but this didn't worked for my multi module android project – ansh sachdeva Mar 09 '22 at 14:47

aggregating gradle multiproject test results using TestReport

6 Answers6

TestReport task

test-report-aggregation plugin

Linked