17

Is there any way to get code coverage using JaCoCo with the tomcat7-maven-plugin embedded instance?

The jacoco-maven-plugin is configured in my WAR's POM to instrument my unit tests, but I'm not sure how to attach the jacoco agent to the embedded Tomcat instance to instrument my integration tests that run against Tomcat. Given that the Tomcat instance is embedded, I'm not sure if this approach is possible. Is there any other way to accomplish this? I can probably switch from using the Tomcat Maven Plugin to using Cargo to get coverage, but I'd prefer to stick with the Tomcat plugin if possible.

Here are a few relevant snippets from my POM:

<plugin>
    <groupId>org.jacoco</groupId>
    <artifactId>jacoco-maven-plugin</artifactId>
    <version>0.6.2.201302030002</version>
    <executions>
        <execution>
            <goals>
                <goal>prepare-agent</goal>
            </goals>
        </execution>
    </executions>
</plugin>
<plugin>
    <artifactId>maven-failsafe-plugin</artifactId>
    <version>2.14</version>
    <executions>
        <execution>
            <id>integration-tests</id>
            <goals>
                <goal>integration-test</goal>
                <goal>verify</goal>
            </goals>
        </execution>
    </executions>
</plugin>
<plugin>
    <groupId>org.apache.tomcat.maven</groupId>
    <artifactId>tomcat7-maven-plugin</artifactId>
    <version>2.1</version>
    <configuration>
        <systemProperties>
            <!-- as expected, this system property doesn't work since Tomcat is embedded, but this is the type of config I'm looking for -->
            <JAVA_OPTS>-javaagent:${project.build.directory}/${jacoco.jar}=destfile=${project.build.directory}/jacoco.exec,append=true</JAVA_OPTS>
        </systemProperties>
    </configuration>
    <executions>
        <execution>
            <id>tomcat-startup</id>
            <goals>
                <goal>run-war-only</goal>
            </goals>
            <phase>pre-integration-test</phase>
            <configuration>
                <fork>true</fork>
            </configuration>
        </execution>
        <execution>
            <id>tomcat-shutdown</id>
            <goals>
                <goal>shutdown</goal>
            </goals>
            <phase>post-integration-test</phase>
        </execution>
    </executions>
</plugin>

Versions: Maven 3.0.4, Tomcat Maven Plugin 2.1, Jacoco 0.6.2.201302030002, Java 7

Donal Fellows
  • 133,037
  • 18
  • 149
  • 215
shelley
  • 7,206
  • 4
  • 36
  • 63
  • Just stumbled onto [MTOMCAT-83](https://issues.apache.org/jira/browse/MTOMCAT-83) which seems to describe this same problem. – shelley Aug 27 '13 at 15:05

6 Answers6

8

I know its been awhile since the question was posted but I don't feel the answer really addressed the root of the problem. Code coverage may work with failsafe or surefire if you are running tests within those plugins. However, if you just want to monitor tomcat with jacoco to get a coverage report current information doesn't provide that. I found that the tomcat7-maven-plugin doesn't allow you to inject the -javaagent for jacoco by simply providing JAVA_OPTS. Switching to cargo I was able to do that like so.

<plugin>
    <groupId>org.jacoco</groupId>
    <artifactId>jacoco-maven-plugin</artifactId>
    <version>0.7.2.201409121644</version>
    <configuration>
        <destFile>${sonar.jacoco.reportPath}</destFile>
        <dataFile>${sonar.jacoco.reportPath}</dataFile>
        <outputDirectory>${project.reporting.outputDirectory}/jacoco-it</outputDirectory>
        <classDumpDir>${project.reporting.outputDirectory}/jacoco-it/classes</classDumpDir>
        <skip>${skipITs}</skip>
    </configuration>
    <executions>
        <execution>
            <id>jacoco-agent</id>
            <phase>pre-integration-test</phase>
            <goals>
                <goal>prepare-agent</goal>
            </goals>
            <configuration>
                <destFile>${sonar.jacoco.reportPath}</destFile>
                <propertyName>jacoco.agent.itArgLine</propertyName>
            </configuration>
        </execution>
        <execution>
            <id>jacoco-report</id>
            <phase>post-integration-test</phase>
            <goals>
                <goal>dump</goal>
                <goal>report</goal>
            </goals>
        </execution>
    </executions>
</plugin>
    <plugin>
    <groupId>org.codehaus.cargo</groupId>
    <artifactId>cargo-maven2-plugin</artifactId>
    <version>1.4.11</version>
    <configuration>
        <skip>${skipITs}</skip>
        <container>
            <containerId>tomcat7x</containerId>
            <zipUrlInstaller>
                <url>http://archive.apache.org/dist/tomcat/tomcat-7/v7.0.16/bin/apache-tomcat-7.0.16.zip
                </url>
                <downloadDir>${project.build.directory}/downloads</downloadDir>
                <extractDir>${project.build.directory}/extracts</extractDir>
            </zipUrlInstaller>
            <dependencies>
                <dependency>
                    <groupId>ojdbc</groupId>
                    <artifactId>ojdbc6</artifactId>
                </dependency>
            </dependencies>
        </container>
        <configuration>
            <home>${project.build.directory}/catalina-base</home>
            <properties>
                <cargo.jvmargs>${jacoco.agent.itArgLine},output=tcpserver,port=6300 -Drunmode=TEST</cargo.jvmargs>
                <cargo.servlet.port>9090</cargo.servlet.port>
            </properties>
            <configfiles>
                <configfile>
                    <file>${basedir}/src/test/conf/context.xml</file>
                    <todir>conf/Catalina/localhost/</todir>
                    <tofile>context.xml.default</tofile>
                </configfile>
            </configfiles>
        </configuration>
    </configuration>
    <executions>
        <execution>
            <id>start-tomcat</id>
            <phase>pre-integration-test</phase>
            <goals>
                <goal>start</goal>
            </goals>
        </execution>
        <execution>
            <id>stop-tomcat</id>
            <phase>post-integration-test</phase>
            <goals>
                <goal>stop</goal>
            </goals>
        </execution>
    </executions>
</plugin>

the important parts are:

  1. <plugin><groupId>org.jacoco</groupId> ...<propertyName>jacoco.agent.itArgLine</propertyName>
  2. <cargo.jvmargs>${jacoco.agent.itArgLine},output=tcpserver,port=6300 </cargo.jvmargs>

When report target is run on the jacoco plugin it will create a directory in ${projectbase}/target/site/jacoco-it/index.html with your coverage report. I use this with the soapui-maven-plugin but it could be used with selenium-maven-plugin also.

Koder
  • 91
  • 1
  • 5
  • Thank you for your reply. As mentioned in my original question, I know that I can accomplish this using cargo (and this has been my temporary workaround), but I would like to know how to accomplish this with the tomcat7-maven-plugin instead. – shelley Jan 27 '15 at 19:52
  • @shelley You can not attach the jacoco agent using the tomcat7-maven-plugin. You should file an issue with that project in this regards. When I did this for my project almost 2 years ago, I had to with to the jetty plugin. Now that does not work with Java 8. Looks like I will be looking into following the method in this answer. – Tim Bender Jun 01 '15 at 21:03
1

You don't need pass JAVA_OPTS to tomcat embedded if you use maven-failsafe-plugin (or maven-surefire-plugin) to run yours integration test. It is because tomcat embedded run in the same process of maven-failsafe-plugin.

So when jacoco-maven-plugin execute prepare-agent it sets argLine that maven-failsafe-plugin uses too.

I created a project to test this, below part of pom:

<build>
    <plugins>
        <plugin>
            <groupId>org.jacoco</groupId>
            <artifactId>jacoco-maven-plugin</artifactId>
            <version>0.6.2.201302030002</version>
            <executions>
                <execution>
                    <goals>
                        <goal>prepare-agent</goal>
                    </goals>
                    <configuration>
                        <includes>
                            <include>my.project.package.only.*</include>
                        </includes>
                    </configuration>
                </execution>
            </executions>
        </plugin>
        <plugin>
            <groupId>org.apache.tomcat.maven</groupId>
            <artifactId>tomcat7-maven-plugin</artifactId>
            <version>2.1</version>
            <executions>
                <execution>
                    <id>tomcat-startup</id>
                    <goals>
                        <goal>run-war-only</goal>
                    </goals>
                    <phase>pre-integration-test</phase>
                    <configuration>
                        <fork>true</fork>
                    </configuration>
                </execution>
                <execution>
                    <id>tomcat-shutdown</id>
                    <goals>
                        <goal>shutdown</goal>
                    </goals>
                    <phase>post-integration-test</phase>
                </execution>
            </executions>
        </plugin>
        <plugin>
            <groupId>org.apache.maven.plugins</groupId>
            <artifactId>maven-failsafe-plugin</artifactId>
            <executions>
                <execution>
                    <id>integration-tests</id>
                    <phase>integration-test</phase>
                    <goals>
                        <goal>integration-test</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
        <plugin>
            <groupId>org.codehaus.mojo</groupId>
            <artifactId>selenium-maven-plugin</artifactId>
            <version>2.3</version>
            <executions>
                <execution>
                    <id>start</id>
                    <phase>pre-integration-test</phase>
                    <goals>
                        <goal>start-server</goal>
                    </goals>
                    <configuration>
                        <background>true</background>
                        <logOutput>true</logOutput>
                        <multiWindow>true</multiWindow>
                    </configuration>
                </execution>
                <execution>
                    <id>stop</id>
                    <phase>post-integration-test</phase>
                    <goals>
                        <goal>stop-server</goal>
                    </goals>
                </execution>
            </executions>
        </plugin>
    </plugins>
Donal Fellows
  • 133,037
  • 18
  • 149
  • 215
Andre Piantino
  • 320
  • 1
  • 8
  • Is the JAVA_OPTS configuration above for the cargo or tomcat plugin? As mentioned in my original question and illustrated in my POM, I've already tried setting JAVA_OPTS for the Maven Tomcat Plugin but don't believe that it works for embedded servers, so I'm not really sure what is being suggested here. – shelley Apr 16 '13 at 13:38
  • I had seen that SO post previously and tested it, but it did not work (as expected, per [this post](http://tomcat.10.x6.nabble.com/Tomcat-7-Maven-Plugin-set-MaxPermSize-td4991157.html), [this issue](https://issues.apache.org/jira/browse/MTOMCAT-83), and given the fact that it's embedded). Are you able to actually run jacoco as demonstrated above with the Maven Tomcat Plugin, or are you just suggesting that the configuration I posted in the initial question _should_ work? – shelley Apr 16 '13 at 16:32
  • 2
    The agent is not even being started. The Tomcat plugin otherwise works as expected; I've been using it this way to run my integration tests for a while now, but have just tried to add jacoco to it. – shelley Apr 19 '13 at 13:50
  • I'm having the same problem!!!! I spent more than 10 hours debugging this. Any resolution? I'm using Gradle (1.5), Embedded Tomcat (7.0.29), Jacoco plugin (0.6.2+), Java 7 and I explicitly see the JVM argument for the javaagent being set. I have tried to use the "tcpserver" version, but there's not port bound to 6300 (default jacoco dump port). Any ideas? – Marcello DeSales Apr 30 '13 at 23:35
  • Hi, I changed my answer because I did an example that works for your problem. I removed the old comments too. – Andre Piantino May 08 '13 at 17:56
  • fork option may create a new JVM but it will not run in the same process as failsafe.. you'd need to pass the argLine to the tomcat process.. – olivervbk Sep 15 '15 at 13:39
0

you can try to take a look at a workaround at this post: http://dougonjava.blogspot.co.il/2013/07/integration-testing-using-maven-tomcat.html

ronapelbaum
  • 1,617
  • 14
  • 19
0

I resolved problems when setting up JaCoCo agent with embedded Tomcat by instrumenting classes offline and then just placing JaCoCo agent on Tomcat classpath (plugin dependency) and adding file jacoco-agent.properties.

I put the working configuration on my blog:

http://burkond.blogspot.de/2014/05/selenium-in-sonar-code-coverage-metrics.html

Ondrej Burkert
  • 6,617
  • 2
  • 32
  • 27
0

I had the exact same problem and the only solution I found was to set the MAVEN_OPTS previously to the Maven build (for example on the command line or in the configuration of a Jenkins job):

export MAVEN_OPTS=-javaagent:~/.m2/repository/org/jacoco/org.jacoco.agent/0.7.4.201502262128/org.jacoco.agent-0.7.4.201502262128-runtime.jar=destfile=./target/jacoco.exec,append=true

This will attach the jacoco agent to the embedded tomcat instance, which will report back the coverage results into the given destfile.

First, it is important that the path to the jacoco runtime JAR is correct. You can manually download and refer to it or use another Maven command to download it into your local .m2 repository:

mvn org.apache.maven.plugins:maven-dependency-plugin:2.8:get -Dartifact=org.jacoco:org.jacoco.agent:0.7.4.201502262128:jar:runtime

Second, make sure that the path to the jacoco.exec file is correct. In my case, I already have an existing jacoco.exec file in the target folder, which contains unit test results. The append=true makes sure that unit and integration tests are combined.

dokaspar
  • 8,186
  • 14
  • 70
  • 98
  • running the entire maven process with the jacoco agent is not recommended. And you'd need to specify the version manually (and referencing your maven repo directly) instead of just using the plugin... – olivervbk Sep 15 '15 at 13:40
0

I managed to do it and it involves some tinkering with finicky stuff:

  • server/container needs to be on a separate jvm that can receive arguments (jacoco-agent). Cargo using embedded containers did not seem to work and was a pain to debug...
  • jvm with jacoco-it needs to stop before the jacoco analysis (duh!) but registering container-stop and jacoco-report on post-integration-test does not guarantee this... (the tcpdump, etc option in a previous answer had this problem)
  • defining random ports for the server/container makes this easy to integrate with continuous integration
  • phantomjs is an extra ;)
  • jacoco should be used as prepare-agent-integration and report-integration for integration-test (does not really make a difference)

Should be run as 'mvn clean verify'

Pom:

<plugin>
    <groupId>org.jacoco</groupId>
    <artifactId>jacoco-maven-plugin</artifactId>
    <version>0.7.4.201502262128</version>
    <executions>
        <!-- unit test coverage -->
        <execution>
            <id>jacoco-pre-unit-test</id>
            <goals>
                <goal>prepare-agent</goal>
            </goals>
            <configuration>
                <destFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</destFile>
                <propertyName>jacoco.ut.argLine</propertyName>
            </configuration>
        </execution>
        <execution>
            <id>jacoco-post-unit-test</id>
            <phase>test</phase>
            <goals>
                <goal>report</goal>
            </goals>
            <configuration>
                <dataFile>${project.build.directory}/coverage-reports/jacoco-ut.exec</dataFile>
                <outputDirectory>${project.reporting.outputDirectory}/jacoco-ut</outputDirectory>
            </configuration>
        </execution>

        <!-- integration test coverage -->
        <execution>
            <id>jacoco-pre-integration-test</id>
            <phase>pre-integration-test</phase>
            <goals>
                <goal>prepare-agent-integration</goal>
            </goals>
            <configuration>
                <destFile>${project.build.directory}/coverage-reports/jacoco-it.exec</destFile>
                <propertyName>jacoco.it.argLine</propertyName>
            </configuration>  
        </execution>
        <execution>
            <id>jacoco-post-integration-test</id>
            <phase>post-integration-test</phase>
             <goals>
                <goal>report-integration</goal>
            </goals>
            <configuration>
                <dataFile>${project.build.directory}/coverage-reports/jacoco-it.exec</dataFile>
                <outputDirectory>${project.reporting.outputDirectory}/jacoco-it</outputDirectory>
            </configuration>
        </execution>
    </executions>
</plugin>
<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>sonar-maven-plugin</artifactId>
    <version>2.6</version>
</plugin>

<!-- Installs PhantomJS so it doesn't have to be pre-installed -->
<plugin>
    <groupId>com.github.klieber</groupId>
    <artifactId>phantomjs-maven-plugin</artifactId>
    <version>0.4</version>
    <executions>
        <execution>
            <!-- should be post-integration-test ? -->
            <phase>test</phase>
            <goals>
                <goal>install</goal>
            </goals>
        </execution>
    </executions>
    <configuration>
        <version>1.9.7</version>
    </configuration>
</plugin>

<!-- Get two free ports for our test server to use -->
<plugin>
    <groupId>org.codehaus.mojo</groupId>
    <artifactId>build-helper-maven-plugin</artifactId>
    <version>1.6</version>
    <configuration>
        <portNames>
            <portName>jetty.port</portName>
            <portName>jetty.port.stop</portName>
        </portNames>
    </configuration>
    <executions>
        <execution>
            <id>reserve-port</id>
            <phase>test</phase>
            <goals>
                <goal>reserve-network-port</goal>
            </goals>
        </execution>
    </executions>
</plugin>

<!-- Run tests (UT) -->
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-surefire-plugin</artifactId>
    <version>2.18.1</version>
    <configuration>
        <argLine>${jacoco.ut.argLine}</argLine>
        <skipTests>${skip.unit.tests}</skipTests>
        <testFailureIgnore>true</testFailureIgnore>
        <excludes>
            <!-- no UT execution, to test only IT
            <exclude>**/<remove this>*Test.java</exclude> -->
        </excludes>
    </configuration>
</plugin>

<!-- Use failsafe to run our integration tests -->
<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-failsafe-plugin</artifactId>
    <version>2.17</version>
    <configuration>
        <systemPropertyVariables>
            <!-- pass these values to the test classes -->
            <phantomjs.binary>${phantomjs.binary}</phantomjs.binary>
            <jetty.port>${jetty.port}</jetty.port>
        </systemPropertyVariables>
    </configuration>
    <executions>
        <execution>
            <id>integration-test</id>
            <goals>
                <goal>integration-test</goal>
                <goal>verify</goal>
            </goals>
        </execution>
    </executions>
</plugin>

<plugin>
    <groupId>org.codehaus.cargo</groupId>
    <artifactId>cargo-maven2-plugin</artifactId>
    <version>1.4.16</version>
    <configuration>
        <skip>${skipITs}</skip>
        <container>
            <containerId>tomcat8x</containerId>
            <zipUrlInstaller>
                <!-- did not work with 'https'. Certificate problem? -->
                <url>http://archive.apache.org/dist/tomcat/tomcat-8/v8.0.26/bin/apache-tomcat-8.0.26.zip</url>
                <downloadDir>${project.build.directory}/downloads</downloadDir>
                <extractDir>${project.build.directory}/extracts</extractDir>
            </zipUrlInstaller>
        </container>
        <configuration>
            <home>${project.build.directory}/catalina-base</home>
            <properties>
                <cargo.jvmargs>${jacoco.it.argLine}</cargo.jvmargs>
                <cargo.servlet.port>${jetty.port}</cargo.servlet.port>
                <!-- <cargo.logging>high</cargo.logging>  -->
            </properties>
        </configuration>
    </configuration>
    <executions>
        <execution>
            <id>cargo-start-tomcat</id>
            <phase>pre-integration-test</phase>
            <goals>
                <goal>start</goal>
            </goals>
        </execution>
        <execution>
            <id>cargo-stop-tomcat</id>
            <phase>integration-test</phase>
            <goals>
                <goal>stop</goal>
            </goals>
        </execution>
    </executions>
</plugin>
olivervbk
  • 276
  • 2
  • 11