3

I am trying to understand what type of test coverage metric Eclipse uses.

I wrote two simple functions as follows:

public class Hello {
  public void f(int a, int b) {
      int sum = a + b;
      if (sum > 0)
          print("blue");
      else if (sum < 0)
          print("red");
      print("done");
  }

  void print(String s) {
      System.out.println(s);
  }


  public void g(int x, int y) {
      if (x == 0 || y > 0) {
          print("red");
      } else {
          print("blue");
      }
  }
}

Then I called the functions from the unit tests like this:

import org.junit.jupiter.api.Test;

class TestHello {

  @Test
  void test_f() {
      new Hello().f(2, 4);
      new Hello().f(-1, -2);
      new Hello().f(-1, 1);
  }

  @Test
  void test_g() {
      new Hello().g(0, 5);
      new Hello().g(5, 0);
      new Hello().g(0, 0);
      //new Hello().g(5, 5);
  }
}

As a result, for the function g, Eclipse says that "1 of 4 branches missed" and marks the line 18 yellow.

Source code line 18 marked yellow

On the other hand, the coverage window shows that the function g is 100% covered while the green bar on the first column is not as long as the function f one where I thought these bars showed how much close you are to 100%.

Coverage shown

So I am trying to make sense out of all of this. Does Eclipse use:

  • branch coverage
  • condition coverage
  • branch and condition coverage
  • modified condition / decision coverage (mc/dc)
  • multiple condition coverage (mcc)
  • something else?

According to %100 coverage shown on the coverage window, it may be using branch and condition coverage. However, the yellow marked line may tell me that it uses something stronger like mc/dc or mcc. Then, there is progress-bar looking-like green bar which I am not sure what is trying to tell me?

I appreciate if someone more knowledgeable than me can make sense out of all these observations and can explain what metric Eclipse is using and what the progress bar means compared to the percentage?

CEGRD
  • 7,787
  • 5
  • 25
  • 35
  • Please note: using screen shots is discouraged (there are many people using screen readers to access websites - and pictures rarely work for these). So only use them as last resort. In other words: consider *extracting* the relevant information from your images and put them into text / table form. I know, that is more work compared to push images here, but again: images are discouraged. – GhostCat Jan 15 '18 at 15:32

3 Answers3

1

I believe it's only statement coverage and branch coverage, nothing else. (AFAIK, there are no code coverage tools out there that actually provide condition coverage.)

And the percentage shown in the coverage window only refers to the first metric (statement coverage), so that's why it's all 100%.

Rogério
  • 16,171
  • 2
  • 50
  • 63
  • Then, why is the bar shorter than the other function's ? Also, if it is branch coverage, why then the line 18 is marked as not covered. According to the branch coverage, it is actually covered because there is only 2 branches not 4 (2 outgoing edges in the control flow graph node); however, Eclipse thinks there are 4 branches. – CEGRD Jan 15 '18 at 15:42
  • Indeed; I don't know what the length of the bars means... it's not the number of instructions; *maybe* it's related to branch coverage. – Rogério Jan 15 '18 at 15:55
1

Eclipse, via the plugin eclemma, uses jacoco for code coverage.

The documentation for jacoco is here.

It allows switching between 'line' and 'branch' coverage, which is defined as follows:

JaCoCo also calculates branch coverage for all if and switch statements. This metric counts the total number of such branches in a method and determines the number of executed or missed branches. Branch coverage is always available, even in absence of debug information in the class files. Note that exception handling is not considered as branches in the context of this counter definition.

If the class files haven been compiled with debug information decision points can be mapped to source lines and highlighted accordingly:

No coverage: No branches in the line has been executed (red diamond)

Partial coverage: Only a part of the branches in the line have been executed (yellow diamond)

Full coverage: All branches in the line have been executed (green diamond)

Community
  • 1
  • 1
soru
  • 5,464
  • 26
  • 30
  • What you are quoting is not "branch coverage" though. For branch coverage, an if statement can only have 2 branches (number of edges coming out of a node in the control flow graph). I am trying to understand what proper class it actually maps to. Also, this does not explain the progress bar. – CEGRD Jan 15 '18 at 15:48
  • Plus, if you want to cover lines which throw an exception: You can't cover these. You can define expectations, but Eclemma will always show you a red diamond in this line, even if you expected the exception. – Chris311 Jan 15 '18 at 15:50
  • @CEGRD In the control flow graph, an `if` statement like `if(cond1 || cond2)` is not only one node with two outgoing edges, because if and only if `cond1` is `false` `cond2` is executed. The same is for expressions with `&&`. Example: `if(x == 0 || throwRuntimeExceptionIfGreaterZero(y))` you have to test with `new Hello().g(5,5)` to get the RuntimeException. – howlger Jan 15 '18 at 16:00
  • @howlger can you provide a source for your way of defining a branch. As far as the resources i find, an if-stmt has always 2 branches no matter how many conditions it has. – CEGRD Jan 15 '18 at 16:18
  • @CEGRD Extract the condition to a separate line: `boolean b = x == 0 || y > 0;` and you will see. For details and more about the Eclipse code coverage tooling I recommend the following video: https://youtu.be/p_mVa9iNmzk – howlger Jan 15 '18 at 16:35
  • @howlger I am trying to map what Eclipse does to a well known test coverage criteria. As I said theoretically speaking, it is definitely not branch coverage. – CEGRD Jan 15 '18 at 16:39
  • @CEGRD It is branch coverage of the bytecode (which can be viewed by the [Bytecode Outline plugin](http://marketplace.eclipse.org/content/bytecode-outline)), not of the source code. Think about `boolean b = condA() || condB()` vs. `boolean b = condA() | condB()`. – howlger Jan 15 '18 at 16:48
0

Without knowing for sure, here is my stab:

  • the bar chart encodes number of instruction by length (would encode non-covered ones with a color)
  • line 18 is highlighted as one branch through the conditional is not tested, as y > 0 is never verified due to shortcut OR; turn it green by uncommenting your line (or running following test new Hello().g(1, 1))
diginoise
  • 7,352
  • 2
  • 31
  • 39