Best practice to run a prebuild step with CMake

Question

Suppose you have a repository with a folder (named dataset) with several .csv files and a python script (named csvcut.py) that takes all the .csv in dataset and generates corresponding .h files.

Those .h files are included in some .cpp files to build an executable (add_executable(testlib...) used for testing.

Suppose you use add_custom_target(test_pattern... to make a target (named test_pattern) that runs csvcut.py, and add_dependencies(testlib test_pattern) to run the script before building testlib.

This works, but it would be better if:

the script was run only when the files in dataset folder or the script itself changes (not when .cpp changes);
the .h files was generated in a subfolder of the build folder (i.e. build/tests/dataset/), and included in the .cpp files like so #include <tests/dataset/generated.h>.

Do you have any suggestions for making these improvements / optimizations?

Thanks, Alberto

The first task seems hard, but the second one seems easy. You can pass `${CMAKE_CURRENT_BINARY_DIR}` to the gererator script, so that it can put the generated files to the build folder. Then add `${CMAKE_CURRENT_BINARY_DIR}` as include directory to the related target (executable or library) with `target_include_directories`. — VictorBian, Apr 09 '21 at 18:24

score 3 · Accepted Answer · answered Apr 09 '21 at 19:01

This requires multiple steps, but can all be handled with standard CMake. First, we'll use add_custom_command to actually generate the files. I'm also adding a custom target, but only since I couldn't figure out how to make an INTERFACE library work without it.

add_custom_command(
    OUTPUT
        "${CMAKE_CURRENT_BINARY_DIR}/include/foo.h"
    COMMAND "${CMAKE_CURRENT_SOURCE_DIR}/gen.py"
    DEPENDS
        gen.py
        foo.h.in
    WORKING_DIRECTORY "${CMAKE_CURRENT_BINARY_DIR}/include"
)
add_custom_target(gen_files
    DEPENDS "${CMAKE_CURRENT_BINARY_DIR}/include/foo.h"
)

For my case, gen.py just spits out a basic header file, but it shouldn't matter. List whatever files you need as output, and your csv file should be under DEPENDS (for me, foo.h.in tries to simulate this).

Since you only mentioned generating header files, I created an INTERFACE library that depends on the gen_files target. I also added the appropriate include directory.

add_library(foo INTERFACE)
target_include_directories(foo
    INTERFACE "${CMAKE_CURRENT_BINARY_DIR}/include"
)
add_dependencies(foo
    gen_files
)

If building a STATIC/SHARED library, I was able to add the generated files directly as sources and dependencies worked, but the INTERFACE library required the extra target (even when I tried listing the files under add_dependencies). Since you already have a custom target, I assume this won't be a huge issue.

Lastly, I have an executable that links against foo.

add_executable(main
    main.c
)
target_link_libraries(main
    PRIVATE
        foo
)

Demo:

$ make clean 
$ ls
CMakeCache.txt  CMakeFiles  cmake_install.cmake  include  Makefile
$ make
[ 33%] Generating include/foo.h
[ 33%] Built target gen_files
[ 66%] Building C object CMakeFiles/main.dir/main.c.o
[100%] Linking C executable main
[100%] Built target main
$ make clean
$ make main
[ 33%] Generating include/foo.h
[ 33%] Built target gen_files
[ 66%] Building C object CMakeFiles/main.dir/main.c.o
[100%] Linking C executable main
[100%] Built target main
$ make
[ 33%] Built target gen_files
[100%] Built target main
$ touch ../foo.h.in 
$ make
[ 33%] Generating include/foo.h
[ 33%] Built target gen_files
[100%] Built target main
$ touch ../gen.py 
$ make
[ 33%] Generating include/foo.h
[ 33%] Built target gen_files
[100%] Built target main
$ ls include/
foo.h

If either the input (foo.h.in) or generation script (gen.py) change, the targets are rebuilt.

Great! Your example was very helpful in solving my problem. For the first time I used the combination of add_custom_command and add_custom_target which could be useful to me in many other scenarios. Thank you! — Alb, Apr 12 '21 at 11:26

Best practice to run a prebuild step with CMake

1 Answers1