14

I have a gradle project(java) in intellij. I right clicked the project in intellij and ran Run Tests in projectName with coverage which created some test report on right hand side. In that right hand side I have numbers like

| Class, %   | Method, %   | Line, %
--------------------------------------
80%(80/100)  50%(100/200)  30%(30/100)

Note: The above numbers are just for example. These are not real.

Now I went to command line and ran gradlew jacocoTestReport which gave a different set of numbers for Method and Line, but Class numbers were same. Why is there a discrepancy in this case?

Is there a way to run intellij's code coverage from command line instead of right clicking?

I just want to know if Intellij uses a different way to calculate these numbers as opposed to jacoco. But even in that case my assumption is that there is only one way to calculate stuff right? Or does intellij or jacoco doesn't count classes that has Lombok annotations etc thereby reducing the number of methods(getters and setters) in the final count?

Michał Krzywański
  • 15,659
  • 4
  • 36
  • 63
theprogrammer
  • 1,698
  • 7
  • 28
  • 48
  • How are the numerators and denominators different from methods and lines? You can easily check methods and lines for IntelliJ. I don't know about Jacoco. I'd believe IntelliJ. I've seen differences caused by the fact that other code coverage tools like SonarCube have to be told explicitly to exclude test classes from the statistics. – duffymo Oct 30 '19 at 14:44
  • 2
    The numbers in my question are just for example. But what's happening is Jacoco has more methods and more lines than Intellij's code runner. I am thinking that Intellij's code runner is not counting lombok annotations etc. If I add @Getter for a class with 2 fields, at runtime this class will have these 2 methods generated. Maybe Jacoco is counting these classes while Intellij is not? Same with other library annotations I guess. – theprogrammer Oct 30 '19 at 14:55

2 Answers2

7

I stumbled over this question on a related mission and although it's old, I figure it may still be interesting to someone.

Method numbers

The main question is if the intellij coverage and jacoco calculate the numbers differently, and which way is correct. Brief answer: The intellij coverage summary uses the methods directly supplied by the developer, while jacoco operates on bytecode level and displays the number of methods found there. To demonstrate this, I created a simple class with four methods:

public class Exp {

    private final LinkedList<Integer> vals = new LinkedList<>();

    public void addVal(int v) {
        vals.add(v);
    }

    public List<Integer> doubled() {
        return vals.stream().map(x -> x*2).collect(Collectors.toList());
    }

    public List<Integer> evens() {
        return vals.stream().filter(x -> x%2 == 0).collect(Collectors.toList());
    }

    public static void main(String[] args) {
        Exp t = new Exp();

        t.addVal(1);
        t.addVal(2);
        t.addVal(3);

        System.out.println(t.doubled());
        System.out.println(t.evens());
    }
}

In the intellij summary displays the following values on the right:

intellij coverage summary

So the number of methods equals the number of methods in the example code. Jacoco reports seven methods, as can be seen in the report (same as in the Emma plugin in eclipse 2020-09): jacoco test report

This is the number of methods we can find in the bytecode, e.g. by using the javap disassembler command. Here we see that the two lambda expressions are implemented as methods of the class, and also a standard constructor is inserted.

C:\_workspace\intellij\Tests\out\production\mod>javap -p Exp.class
Compiled from "Exp.java"
public class Exp {
  private final java.util.LinkedList<java.lang.Integer> vals;
  public Exp();
  public void addVal(int);
  public java.util.List<java.lang.Integer> doubled();
  public java.util.List<java.lang.Integer> evens();
  public static void main(java.lang.String[]);
  private static boolean lambda$evens$1(java.lang.Integer);
  private static java.lang.Integer lambda$doubled$0(java.lang.Integer);
}

What puzzles me a little is that the intellij coverage report (Run->Generate Corevage Report) displays five methods:

intellij coverage report

Adding a standard constructor to the code and re-generating the report reveals that the report included the generated standard constructor, but not the lambda expressions. There seems to be an intermediate counting method.

As for the question if intellij or jacoco is right I would say that they both are, it's just a question of definition.

Line numbers

In my tests all reports displayed consistent line numbers. In the example above 13 lines containing executable code were reported. My impression of the intellij line count in the coverage summary is that it does not refresh properly all the time. A clean rebuild may have been necessary.

jf_
  • 3,099
  • 2
  • 13
  • 31
  • I would add only that with bigger files the coverage numbers will be different, and a dev team using Eclipse and Intellij will be puzzled what is the coverage. I have an SDK where jacoco/Eclipse says coverage is 68% while IntelliJ says coverage is 80+%. It is just bad and annoying. Tools like Sonar and Codecov parse jacoco report if I'm right, so IntelliJ coverage just useless. – AndrasCsanyi Jul 07 '21 at 14:30
-1

I noticed that the @Test annotation I was using came from org.junit.jupiter.api.Test which wasnt being picked up by Jacoco. Changing the import to import org.junit.Test fixed my coverage issue.

  • 7
    Changing from `org.junit.jupiter.api.Test` to `org.junit.Test` means that you switched from JUnit 5 to JUnit 4, which is unlikely to be the proper solution. – Mark Rotteveel Jan 04 '21 at 17:30