2

When I run gcovr after running my unit test suite I get wrong number for coverage of header files where some inline member function definitions are placed. For example:

----------------------------------------------------------------------------  
File                                       Lines    Exec  Cover   Missing  
------------------------------------------------------------------------------   
include/analysis/dataobjects/DetailedHit.h     6       2    33%   41-43,45
include/analysis/dataobjects/Hit.h            19       0     0%   31-33,35-36,42-43,50-51,58-59,74-75,82-83,90-91,98-99

For Hit.h the reported coverage is 0%, but I'm sure that at least some code from that header is executed during the unit test run since if I place a cout in it then I see it in the console ouptut. On the web it is often suggested that it is a problem coming from the fact that the compiler inlines the member function code, so that no function call is generated and thus the coverage tools do not track the execution. So I added the flags:

-fno-inline -fno-inline-small-functions -fno-default-inline

to the invocation of the compiler (gcc 8.2.1) but I get the same coverage report. So I don't understand what's happening.

The thing that mostly puzzles me is why gcovr reports 2 covered lines in DetailedHit.h. This header is very similar to Hit.h so I'd expect the same behavior with 0% reported coverage, but this member function:

const std::vector<Herd::ParticleHit> ParticleHits() const {
  return _particleHits;
}

results to be executed 10 times. It is a simple function so it should be inlined as the ones in Hit.h, still it results to be covered. In case it matters, Hit is a concrete base class for DetailedHit, and both do not have virtual methods.

I think I'm missing some important piece of knowledge about how gcov and gcovr work, but I couldn't find a relevant clue on the web so I'd appreciate any help on this. Thanks.

Nicola Mori
  • 777
  • 1
  • 5
  • 19
  • 1
    Please show the code from both headers to reduce the necessary guessing. – Yunnosch Mar 08 '19 at 07:45
  • 1
    Make a [mcve] to allow reproducing your observation. – Yunnosch Mar 08 '19 at 07:45
  • @Yunnosch: you can get the two headers from https://basket.fi.infn.it:443/f/4fad91381d/?dl=1 and https://basket.fi.infn.it:443/f/b59cf1a5ac/?dl=1 I'm working to create a reproducer but I'm finding some difficulties. I'll post it as soon as I get it working. Thanks. – Nicola Mori Mar 08 '19 at 08:06
  • Please add all helpful information directly here. It improves your chances for helpful information if you reduce the effort needed for giving it. – Yunnosch Mar 08 '19 at 08:07
  • Could you generate the `--html-details` report instead of the default text report? The HTML files will annotate the source code with more detailed coverage status. Could you post a screenshot of that? Have you disabled optimizations for your tests? If you are sure that the functions are executed but not detected as covered this could be due to things gcovr's lack of full support for template specialization coverage with GCC 8. – amon Mar 09 '19 at 12:20
  • @amon I compile using the CMake Debug profile, which adds just the -g flag. No -O whtasoever. I created the html and took a screnshot of the relevant lines. You can find it here: https://basket.fi.infn.it:443/f/c8c60be962/?dl=1 – Nicola Mori Mar 09 '19 at 17:49
  • @Yunnosch I can't reproduce the issue with a simple reproducer. Every reproducer I create report the coverage correctly. So I looked at the assembly generated by my original code and by the reproducer, and I found a slight difference around the call to the Hit::EDep function (whose coverage is not correctly reported). They both seem to do an operation on a memory address mapped on the gcov space, but these operations are different (a lock addq for the original program and a mov for the reproducer). Sorry for the scarce details but I don't understand well what's going on. – Nicola Mori Mar 09 '19 at 17:53
  • @NicolaMori Debug mode sounds good. In the HTML report you can click on the filenames to see per-line coverage data. A screenshot of the two reports would be helpful. – amon Mar 09 '19 at 17:54
  • @amon here are the links for the html pages for the two headers: https://basket.fi.infn.it:443/f/6ce4e00ef1/?dl=1 https://basket.fi.infn.it:443/f/522127a2b4/?dl=1 – Nicola Mori Mar 09 '19 at 18:00
  • @NicolaMori The reports seem to indicate that gcov/gcovr is working as expected and that the functions aren't executed. Some functions like SetSNRatio() aren't even marked as uncovered, which means that they are never called in the compilation unit. One possible cause is that the CMake build system causes headers to be included under different paths and gcovr ends up being confused. This could be verified by posting the output of gcovr in `--verbose` mode. There may be relevant fixes in the development version of gcovr which you can pip-install directly from GitHub. – amon Mar 09 '19 at 18:48
  • Analyzing the assembly I noticed that the code of the library that contains Hit code is actually instrumented with coverage calls (also for Hit::EDep). However, Hit::EDep, being inline, is compiled also when I compile the test program, and in this case I do not pass --coverage to the compiler since I'm not interested in coverage for the test code. So Hit::EDep is not instrumented, and incidentally all the calls (also those from the library) seems binded to this non-instrumented version when I run my tests. Is there a way to fix this without instrumenting the whole test program? – Nicola Mori Mar 11 '19 at 15:04

1 Answers1

3

The methods with 0% coverage were inline methods in a header.

I did compile the library I was testing with gcc --coverage which instruments the machine code to collect coverage. However, I did not instrument the test executable since I am not interested in the coverage of the tests themselves. So the test executable contained code for the inline methods without instrumenting them.

As a result, the methods were known to gcov but no coverage was collected (only the non-instrumented inline version was executed).

The solution:

  • also instrument the tests by using the --coverage compiler flag
  • filter out the unwanted coverage data of the tests with the -e/--exclude gcovr option
amon
  • 57,091
  • 2
  • 89
  • 149
Nicola Mori
  • 777
  • 1
  • 5
  • 19
  • It's great that you found the problem! I rewrote the answer to make it easier for others to understand. I don't think the linker played a role here: multiple “instances” of an inlined function can coexists, the ODR does not apply to them. Instead, the instrumented version caused gcov to *know* about the inlined functions, but the tests only executed a non-instrumented version. Therefore, no coverage was collected :( – amon Mar 11 '19 at 18:57
  • @amon thanks for making my answer more understandable. I understand your point about the linker, but there's one thing that still sounds strange to me. In the library there is code that call the instrumented version of the method (placed in the same library) when I run the main executable, however the same code calls the non-instrumented version (placed in the test library) when I run the test executable. Why? The code that calls the method is the very same in both cases, it has been compiled in a shared library just once and linked both to the main executable and to the test executable. – Nicola Mori Mar 13 '19 at 07:03
  • Hmm, that could indeed point to linker shenanigans. Inline functions give the compiler and linker loads of undefined behaviour to work with. The compiler may or may not generate code for an inline function, and if there is a conflict the linker is free to discard any version, use both, or do whatever. This case was relatively benign, but in general you should use the same compiler options for all code that you link together. – amon Mar 13 '19 at 11:21