0

I have a regression test suite consisting of multiple custom targets created with add_custom_target(). Moreover there is a "convenience" target regressions to run all regressions. It simply contains all single regression targets as dependency. This is reflected in the following MCVE:

cmake_minimum_required(VERSION 3.17)

project (Regressions)

add_custom_target(reg_1 COMMAND ${CMAKE_COMMAND} -E echo 'First regression')
add_custom_target(reg_2 COMMAND ${CMAKE_COMMAND} -E echo 'Second regression')
# ...
add_custom_target(regressions DEPENDS reg_1 reg_2)

Now I can run cmake --build . --target regressions from the build directory and reg_1 and reg_2 are run as part of regressions. My problem is that if one of the regressions fail, the remaining are not executed. But of course I want to always run all regressions and only have a summary of the failed ones. How can I achieve this behavior, i.e. always execute all subtargets, no matter whether some of them fail?

I assume that the natural way to do this is to use add_test() (after all regressions runs are tests), but I failed because the custom targets are no executables and AFAIK you cannot use custom CMake targets with add_test().

Please feel free to recommend an alternative to my current approach. If I could handle everything using ctest that would be preferred anyway.

andreee
  • 4,459
  • 22
  • 42
  • 1
    This really looks like tests and you could use `ctest`. `custom targets are no executables and AFAIK you cannot use custom CMake targets with add_test()` ? just then `add_test(COMMAND ${CMAKE_COMMAND} --target custom_target)`. You seem to be asking XY question. Your real question seem to be how to use custom cmake target in `add_test()`. – KamilCuk Jul 08 '20 at 10:36
  • Gosh! Seems like I'm too stupid today. Thanks @KamilCuk, if I can run targets by calling CMake, of course I can run the appropriate tests... – andreee Jul 08 '20 at 11:22

1 Answers1

2

Thanks to @KamilCuk's answer, I realized my problem of not being able to add custom targets as tests is not really a problem.

I can invoke CMake with add_test(), and the cmake command can run custom targets.

Adding:

enable_testing()
add_test(NAME regression1 COMMAND ${CMAKE_COMMAND} --build ${CMAKE_BINARY_DIR} --target reg_1)
add_test(NAME regression2 COMMAND ${CMAKE_COMMAND} --build ${CMAKE_BINARY_DIR} --target reg_2)

to the code above solves the problem. Additionally, instead of depending on the single custom targets, regression can simply invoke ctest -R "regression*" to invoke all (and only) regression targets, in case other tests exist in the CMake project, like this:

add_custom_target(regressions COMMAND ${CMAKE_CTEST_COMMAND} -R "regression*")
andreee
  • 4,459
  • 22
  • 42