Sonarqube overall coverage does not match jacoco report coverage

Question

I have a multi-module Android project and I'm seeing a discrepancy between the coverage that jacoco reports and what Sonarqube reports. The project is also a multi-flavor project that generates a lot of different variants. I am using this plugin to help me generate all of the tasks. The tasks generate an individual report for each variant.

When I run my jacoco reports I see this:

When I run the sonar scanner i see this:

I have some exclusions on my project,but even without them the coverage %s don't match.

I feel like I may not be providing the same bytecode as hinted in this question:

Here is my relevant info: Sonar Version 5.6.

Gradle runner

plugins { id "org.sonarqube" version "2.0.1" }

Sonar config: (on root build.gradle)

sonarqube {
    properties {
        property "sonar.projectKey", "com.xxx.myApp"
        property "sonar.projectName", "Android My App"
        property "sonar.projectVersion", "3.0"
        property "sonar.java.binaries", "build/classes"
        property "sonar.coveragePlugin", "jacoco"
        property "sonar.jacoco.reportMissing.force.zero", "false"
    }
}

Sonar config (on app/build.gradle)

sonarqube {
    properties {
        property "sonar.sources", "src/main/java"
        property "sonar.tests", "src/test/java"
        property "sonar.java.tests", "src/test/java"
        property "sonar.junit.reportsPath", "build/test-results/myAppGoogleMobileDebug"
        property "sonar.java.binaries", "build/intermediates/classes/myAppGoogleMobile/debug"
        property "sonar.jacoco.reportPath", "build/jacoco/testMyAppGoogleMobileDebugUnitTest.exec"
        property "sonar.coverage.exclusions", coverageExclusions

    }
}

Jacoco config on (app/build.gradle)

def coverageExclusions = ['**/AEWatchApp.*', '**/**_Factory.*',
                          '**/QaSettingsActivity.*',
                          'com/aetn/android/tveapps/activities/**',
                          'com/aetn/android/tveapps/test/**',
                          'com/aetn/android/tveapps/app/injection/modules/**',
                          'com/aetn/android/tveapps/app/injection/components/**',
                          'com.aetn.android.tveapps.mock/**',
                          'com/aetn/android/tveapps/databinding/**']


jacocoAndroidUnitTestReport {
    csv.enabled false
    html.enabled true
    xml.enabled true
    excludes += coverageExclusions
}

Godin · Accepted Answer · 2021-11-16T12:01:19.780

25

As far as I can see branch coverage is the same: 40% in both cases, 15 uncovered.

And comparison of "instructions" (shown in screenshot of JaCoCo report) with anything else is like comparison of apples and oranges - they don't represent the same thing. See http://www.eclemma.org/jacoco/trunk/doc/counters.html about counters that JaCoCo provides. And https://docs.sonarqube.org/latest/user-guide/metric-definitions about what SonarQube shows. Instructions coverage is presented only in JaCoCo.

"lines" ("27.1%" shown in screenshot of SonarQube) is not the same as "instructions": Single line of code usually contains many bytecode instructions. So for example if in total you have 100 instructions in 10 lines and cover 1 line with 20 instructions, then missed instructions 80%, but missed lines 90%.

So all in all, there is no discrepancy or at least it is not shown on your screenshots.

edited Nov 16 '21 at 12:01

answered Oct 31 '16 at 12:59

Godin

9,801
2
39
76

thanks for your answer. Can you please clarify? What then is the 28% coverage that Sonar shows? Notice the large 28.8% "coverage" on the Sonar report. I don't quite understand what that represents and how it relates to the jacoco report. Is branch coverage what I should be concerned with and not missed instructions? – Nelson Ramirez Oct 31 '16 at 13:58
4

"28.8%" - is a mix of Line coverage and Branch coverage, see page http://docs.sonarqube.org/display/SONAR/Metric+Definitions about how it is computed, JaCoCo does not compute this. "27.1%" - is Line coverage, which should be presented in JaCoCo report, but not shown on your screenshot (eg see 5 column "missed/lines" at http://www.eclemma.org/jacoco/trunk/coverage). "concerned"? what do you mean? The initial question is about discrepancy and how to use code coverage metrics is a different story. SonarQube provides aggregation of branch and lines to simplify monitoring of both at same time. – Godin Oct 31 '16 at 18:14
thanks for pointing me to metric definitions. That helps a lot. – Nelson Ramirez Nov 01 '16 at 18:57
2

The sonarqube metrics definitions link seems to be broken. Here is a working link: https://docs.sonarqube.org/latest/user-guide/metric-definitions/ – juancho85 Sep 13 '21 at 13:27

Sonarqube overall coverage does not match jacoco report coverage

1 Answers1

Linked

Related