0

I am creating a generic C++, CMake, and Catch project template that I plan on using in the future, and want to enable code coverage reports for it. To do so, I decided to add the following CMake module into my list of Modules: CodeCoverage.cmake. My usage for this essentially boils down to the following snippet:

if (ENABLE_COVERAGE AND NOT CMAKE_CONFIGURATION_TYPES)
    if (NOT CMAKE_BUILD_TYPE STREQUAL "Debug")
        message(STATUS "Coverage in a non-debug build may lead to misleading results! Setting build type to 'Debug'!")
        set(CMAKE_BUILD_TYPE "Debug" CACHE STRING "" FORCE)
    endif (NOT CMAKE_BUILD_TYPE STREQUAL "Debug")

    include(CodeCoverage)
    APPEND_COVERAGE_COMPILER_FLAGS()

    if (NOT BUILD_TESTS)
        message(STATUS "Tests must be enabled in a coverage build! Setting BUILD_TESTS to On!")
        set(BUILD_TESTS On CACHE STRING "" FORCE)
    endif (NOT BUILD_TESTS)

    set(COVERAGE_EXCLUDES 'tests/*')

    include(CodeCoverage)
    APPEND_COVERAGE_COMPILER_FLAGS()

    SETUP_TARGET_FOR_COVERAGE(NAME coverage EXECUTABLE tests DEPENDENCIES coverage)
else (ENABLE_COVERAGE AND NOT CMAKE_CONFIGURATION_TYPES)
    # ...
endif (ENABLE_COVERAGE AND NOT CMAKE_CONFIGURATION_TYPES)

Details.

  • ENABLE_COVERAGE is command line flag which I use to enable/disable code coverage, which will be generated using lcov/gcov.
  • tests is the name of my executable which runs all of my unit tests. First I use add_subdirectory to add its folder, and inside the folder I use:

    set(TEST_SOURCES test_hello_world.cpp test_factorial.cpp)
    
    add_executable(tests main.cpp ${TEST_SOURCES})
    target_link_libraries(tests Project-Name-lib)
    
    include(ParseAndAddCatchTests)
    ParseAndAddCatchTests(tests)
    

My problem:

When I usually run my make script, the code compiles with ease, including the test executable which takes ~10 seconds to build. However, when I enable coverage for my code, the code gets stuck on building main.cpp.o (Inside tests), and make never finishes. When I run make VERBOSE=1, I get the following command which is being run for said object:

cd project_dir/build/tests && /usr/bin/g++-5 
    -g -O0 --coverage -fprofile-arcs -ftest-coverage -g -O0 
    --coverage -fprofile-arcs -ftest-coverage -g 
    -I/project_dir/include -I/project_dir/build    
    -std=c++14 -o CMakeFiles/tests.dir/main.cpp.o -c /project_dir/main.cpp

(I have shortened some of the paths for readability and indented lines are part of the same command)

Apparently, even with set(COVERAGE_EXCLUDES 'tests/*'), the compiler is still appending the coverage flags to the test executable, which I think is the source of the slow build time. If my conclusion is correct, then how can I tell the compiler to not add these flags to my test executable? If my conclusion is incorrect, then how can I reduce the build time for this executable? (I don't want to directly mention flag names, since that might not work across all compilers(By which I mean GCC/Clang))

On a side note, my main.cpp contains:

#define CATCH_CONFIG_MAIN
#include "catch.hpp"

And a sample test file contains:

#include <project-abbr/factorial.hpp>
#include "catch.hpp"

TEST_CASE("function factorial")
{
    SECTION("normal arguments")
    {
        REQUIRE(factorial(2) == 2);
        REQUIRE(factorial(3) == 6);
        REQUIRE(factorial(4) == 24);
        REQUIRE(factorial(5) == 120);
    }
}
Arnav Borborah
  • 11,357
  • 8
  • 43
  • 88
  • `I don't want to directly mention flag names, since that might not work across all compilers` - The script you use for coverage works only with gcc and clang, which has identical coverage flags. So it wouldn't be a problem to remove this flags. But because the script sets compiler flags into `CMAKE_CXX_FLAGS_*` variable, the flags are used for **every compilation unit**. So the only way for build tests without that flags is building them **outside of the main project**. E.g., with `ExternalProject_Add`. – Tsyvarev Mar 12 '18 at 23:12
  • @Tsyvarev that may well be true, but first I need to confirm if even the problem I found is correct. Is it the flags that are causing the build slowdown (to the point where it doesn't compile) or not? Also, how would you suggest I implement this? With cross compiler I did mean GCC/Clang, so I guess mentioning specifics is no longer a problem! It is just coverage, not the whole project anyways, so its ok for such a solution. – Arnav Borborah Mar 13 '18 at 00:55
  • `Is it the flags that are causing the build slowdown (to the point where it doesn't compile) or not?` - Coverage options *definitely* slow the compilation. As for "freezeing" the compilation, nothing should normally do that. So what do you observe is likely a compiler bug and should be reported accordingly (not here). – Tsyvarev Mar 13 '18 at 08:22
  • @Tsyvarev That sounds like too much effort for now. I might consider filing a bug report in the future. For now, could you post an answer explaining how I can remove the coverage flags from my executable `tests`. – Arnav Borborah Mar 13 '18 at 11:12

1 Answers1

0

UPDATE

The following code isn't really a solution. It works only for all functions that are defined only in source files. Meaning if you had a template library or a header only library, then this solution is completely useless. The true solution is to call the function append_coverage_compiler_flags which allows for complete coverage. However, this may cause a compilation freeze, as shown in the OP's post...


Ok, I experimented a bit, looked at the documentation, and found a solution which isn't really optimal but works nonetheless. I used the function target_compile_options to only add the coverage flags to the targets I want instead of the function ADD_COVERAGE_COMPILE_FLAGS() from the module, as follows:

if (NOT CMAKE_BUILD_TYPE STREQUAL "Debug")
    message(STATUS "Coverage in a non-debug build may lead to misleading results! Setting build type to 'Debug'!")
    set(CMAKE_BUILD_TYPE "Debug" CACHE STRING "" FORCE)
endif (NOT CMAKE_BUILD_TYPE STREQUAL "Debug")

include(CodeCoverage)

if (NOT BUILD_TESTS)
    message(STATUS "Tests must be enabled in a coverage build! Setting BUILD_TESTS to On!")
    set(BUILD_TESTS On CACHE STRING "" FORCE)
endif (NOT BUILD_TESTS)

set(COVERAGE_EXCLUDES 'tests/*')

include(CodeCoverage)

include_directories(include ${PROJECT_BINARY_DIR})
add_subdirectory(src)

target_compile_options(Project-Name-lib PRIVATE "--coverage")
target_compile_options(Project-Name PRIVATE "--coverage")

SETUP_TARGET_FOR_COVERAGE(NAME coverage EXECUTABLE tests DEPENDENCIES coverage)

There is a caveat to this solution. Apparently, header files are not included in the code coverage...

Arnav Borborah
  • 11,357
  • 8
  • 43
  • 88