Testing our CMake Modules

Question

We created multiple additional functions for CMake. They became quite a lot, and we need to (unit) test them.

There are simple ones, that are only variable bases like:

function(join_list LIST GLUE)

These can be tested with a custom CMake Script, that checks the results. For this we also wrote a set of assert-macros.

This becomes way harder when the functions are target based:

function(target_my_custom_property_set TARGET VALUE)

We need a multiple CMakeLists.txt Files that need to be configured. Configuration must succeed or fail with specified messages. Also the result files must be checked.

I wonder, is there an easier way? Is there a existing framework? How does Kitware test the shipped modules?

score 2 · Answer 1 · answered Dec 24 '18 at 02:09

ctest is the framework for running all sorts of tests. There are many tests for cmake that get run as part of the CMake Testing Process. These tests are part of the source code in the Tests folder and are part of CMakeLists.txt.

The specific tests you want to look at are located in the RunCMake folder. These tests utilize RunCMake.cmake. A good example is the tests in message. What these tests do is utilize execute_process to capture the output from cmake and compare the output from the cmake configure step to the contents for a file with the expected output. The return value from cmake is also returned and can be tested.

You don't specify what "results files" are. There are examples that are more complicated that the perform a configuration and build and scan some files to verify there contents.

It may be easier if you separate out checking messages in a failed configure vs a failed build vs a passing configure and build and a specific output message.

score 0 · Answer 2 · answered Mar 16 '23 at 09:33

More then 4 years passed and I still struggle with this. However, one solution we use is to mock the target based functions:

function(set_target_properties)
   assert(${CMAKE_ARGC} == 8)
   # more asserts....
endfunction()

target_my_custom_property_set(tar val)

This often is enough to test those functions.

Testing our CMake Modules

2 Answers2