I'm developing a Gradle plugin and I'm trying to configure my project to let me get code coverage metrics on it. I have unit and integration tests based on the Spock framework.
I've tried using both Jacoco and Cobertura to analyse my project. Here is the configuration I'm working with:
Gradle: 2.2.1
Groovy: 2.3.6
Ant: Apache Ant(TM) version 1.9.3 compiled on December 23 2013
JVM: 1.8.0_25 (Oracle Corporation 25.25-b02)
OS: Mac OS X 10.10.1 x86_64
I'm using the gradle-cobertura-plugin v2.2.5.
Cobertura
In the case of Cobertura, my project's reported line coverage is only 35%. There are large sections of code that I have written tests for that are reported as not tested by Cobertura:
In particular, Cobertura reports no coverage for the nested static class Version.Parser despite there being a complete Spock specification devoted to this:
package com.github.tagc.semver
import spock.lang.Specification
import spock.lang.Unroll
import com.github.tagc.semver.Version.Parser
@Unroll
class VersionParserSpec extends Specification {
private static final Parser PARSER = Version.Parser.getInstance()
def "Version information should be extracted from files if parsing is not strict"() {
given:
def versionFileText = "version='$versionString'"
expect:
PARSER.parse(versionFileText, false) == version
where:
versionString | version
'0.1.2-SNAPSHOT' | new Version(0,1,2,false)
'1.2.4' | new Version(1,2,4,true)
'1.3-SNAPSHOT' | new Version(1,3,0,false)
'0.4' | new Version(0,4,0,true)
}
def "Valid version representation should be parsed successfully"() {
expect:
PARSER.parse(input, true) == version
where:
input | version
'0.1' | new Version(0,1,0,true)
'1.3-SNAPSHOT' | new Version(1,3,0,false)
'1.1.1' | new Version(1,1,1,true)
'0.2.7' | new Version(0,2,7,true)
'0.4.9-SNAPSHOT' | new Version(0,4,9,false)
'6.3.16-SNAPSHOT' | new Version(6,3,16,false)
' 1.2.3-SNAPSHOT' | new Version(1,2,3,false)
' 1.3.5-SNAPSHOT ' | new Version(1,3,5,false)
}
def "Invalid version representation (#input) should cause an exception to be thrown"() {
when:
PARSER.parse(input, true)
then:
thrown(IllegalArgumentException)
where:
input << [
'1.2.a',
'1,2,3',
'2.4.-1',
'3-4-9',
'1.4.5-SNPSHOT',
'1.4.5-SNAPSHOTasd'
]
}
}
Below are the relevant parts of my Gradle build script:
buildscript {
repositories { jcenter() }
dependencies {
// Cobertura plugin
classpath "net.saliman:gradle-cobertura-plugin:2.2.5"
}
}
configurations.all {
resolutionStrategy {
force 'org.ow2.asm:asm:5.0.3'
forcedModules = [ 'org.ow2.asm:asm:5.0.3' ]
}
}
apply plugin: 'net.saliman.cobertura'
check.dependsOn 'cobertura'
cobertura {
coverageFormats = [ 'html', 'xml' ]
}
Jacoco
In comparison, Jacoco reports a much more plausible coverage of 68% (by instructions).
Coverage of the same Version.Parser section is reported as this:
The relevant parts of my build script are:
apply plugin: "jacoco"
task integrationTest(type: Test) {
description = 'Runs the integration tests.'
group = 'verification'
testClassesDir = sourceSets.integrationTest.output.classesDir
classpath = sourceSets.integrationTest.runtimeClasspath
jacoco {
destinationFile = file("$buildDir/jacoco/integrationTest.exec")
classDumpFile = file("$buildDir/classes/integrationTest")
}
}
jacocoTestReport {
executionData test, integrationTest
reports {
xml.enabled true
html.enabled true
}
}
Since Jacoco seems to be working fine, I'd ideally like to just stick with it. However, Sonar doesn't seem to work properly with Jacoco when writing code in Groovy, so I seem to be stuck with Cobertura. Is there any reason why Cobertura could be giving me these coverage results?
EDIT
I have raised this as an issue on the Gradle Cobertura plugin Github repository.
