11

I recently used EclEmma plugin in eclipse to find code coverage of JUnits in my application and it worked great.

In the next step, I want to be able to find code coverage for end-to-end tests / functional tests which are called on our application from outside(they reside on a separate server).

Basically, we package our application as a jar (we use maven) and deploy it on a server. And we can trigger the functional tests at this location.

Is there a way to find code coverage in this case?

My understanding of how code coverage tool works (in raw language) is that it loads the classes and generates a report based on the part of code that has been hit. So as per this understanding, I don't need to have access to the test-code. I only need to somehow plug the code coverage tool into my application jar and whenever any code in this jar is called, report will be updated. Is my understanding correct?

NOTE: I am open to use other code coverage tool if this is possible with other tools.

tryingToLearn
  • 10,691
  • 12
  • 80
  • 114
  • 1
    I believe your understanding of "external" tests is flawed. Code coverage is a developer test coverage metric. QA testing should test the code against requirements; basically "this input generates this activity in the software as demonstrated by this output". – DwB May 04 '18 at 11:59
  • Point is we want to be sure that maximum part of our code has been touched by tests regardless of the nature of tests before shipping to client. – tryingToLearn May 04 '18 at 12:02
  • Then perform unit tests to cover the code. – DwB May 04 '18 at 12:02
  • 1
    @DwB A code coverage report from higher level tests -- even from manual tests or even from a period of real-world usage -- is informative and useful. You shouldn't use the coverage % as a metric. But you can eyeball the reports and see stuff 'Hey, why does that method never get executed?'; 'Ah, look, our feature switch is working', 'Hmm, that bit should never trigger' etc. – slim May 04 '18 at 12:08
  • @slim that pretty much sums up my intentions – tryingToLearn May 04 '18 at 13:06

3 Answers3

15

You can run your code on a server, instrumented at runtime by the JaCoCo agent, by adding the agent to the Java command line. For example if your process is currently launched with:

java -jar myApp.jar

You can change it to 

java -jar myApp.jar -javaagent:/some/path/jacocoagent.jar

By default this will write coverage data to the file jacoco.exec when the VM terminates, but you can override this with options, enabling you to get coverage data over TCP/IP. See the JaCoCo Agent docs.

You can format this into a report (e.g. HTML) using:

java -jar jacococli.jar report jacoco.exec [options]

See the JaCoCo CLI docs for options.

... or you can use EclEmma to analyse the output.

If there are genuine reasons you can't use the agent, as a last resort you can statically convert your class files into instrumented class files, using the instrument command in jacocococli. See the JaCoCo 'Offline Instrumentation' documentation for information about this.

Directly invoking the agent and using jacococli.jar are the most basic ways of using JaCoCo. Since you are using Maven, you can get many of the same effects using the JaCoCo Maven plugin.

Godin
  • 9,801
  • 2
  • 39
  • 76
slim
  • 40,215
  • 13
  • 94
  • 127
  • Does that mean there is no need to hook jacoco during compile time? It can be hooked on the fly? – tryingToLearn May 04 '18 at 11:50
  • Yes; I'm not sure how the answer could make that any clearer. – slim May 04 '18 at 11:51
  • But in my case jar is not executable jar and functional tests themselves call the respective part of code. That's why I was looking for a way to hook jacoco at compile time. – tryingToLearn May 04 '18 at 11:53
  • 2
    You still launch Java somewhere though. If you're running in Tomcat, for example, add the agent to the `java` command in the Tomcat startup scripts. – slim May 04 '18 at 11:58
  • Thanks. I tried this and it worked with a few tweaks. The report that was generated was not viewable directly in **Eclemma** i.e. no covergae was shown. (It would be great if I can make that work). Even providing the **class files** created by maven that it used to generate jar didn't work. (jacoco-cli gave some warning like execution env different) I had to **extract application.jar** and provide the path of extracted class files to get the report. Am I missing something here or is this the way it should be? – tryingToLearn May 08 '18 at 11:22
1

This is how i did it with maven in a jenkins pipeline 

mvn clean org.jacoco:jacoco-maven-plugin:prepare-agent test -Dmaven.test.failure.ignore=true

This will create a target/jacoco.exec that has the code coverage data.

Obviously we cannot interpret the output, but tools and plugins like SonarQube can do that.(In my case i used sonarqube)

You can however use below to generate into html/csv format which will be located in target/site/jacoco/index.html , target/site/jacoco/jacoco.csv

mvn org.jacoco:jacoco-maven-plugin:report OR

java -jar jacococli.jar report jacoco.exec [options]

Alternatively, you can refer to this tutorial which did all this in the pom.xml

codereal
  • 150
  • 1
  • 11
1

I had the same issue. This is how I fixed it locally. By adding the jacoco agent in the vm args. In addition, after jacoco version 6.1, the exec file is created at the beginning (blank) and then later populated after the server is gracefully shutdown which apparently eclipse doesn't do. So we get an empty .exec file. The key is to add output=tcpserver and then import the code coverage.

-javaagent:C:\Users\JohnDoe\Downloads\jacoco-0.8.5\lib\jacocoagent.jar=output=tcpserver.

Import the results into Eclipse. This is done from File -> Import -> Coverage Session -> select Agent address radio button but leave defaults -> enter some name and select code under test. Default should be 127.0.0.1 with port 6300.

Mayur
  • 61
  • 1
  • 3