1

I have project name libtld where I first create a tool¹:

project(tld_parser)
add_executable(${PROJECT_NAME}
    ../tools/tldc.cpp

    tld_compiler.cpp
    tld_file.cpp
    tld_strings.c
)

Then I use that tool to generate the tld_data.c file, which is a table with all the TLDs (in my newer version of the library, this is a copy of the RIFF/TLDS binary file which is to be compiled internally as a fallback). Here is the add_custom_command() I use to generate said file:

project(tld_data)
set(TLD_DATA_C ${PROJECT_BINARY_DIR}/tld_data.c)
file(GLOB_RECURSE TLD_FILES ${CMAKE_SOURCE_DIR}/conf/tlds/*.ini)
add_custom_command(
    OUTPUT ${TLD_DATA_C}
    COMMAND "tld_parser"
                "--source"
                    "${CMAKE_SOURCE_DIR}/conf/tlds"
                "--verify"
                "--output-json"
                    "--include-offsets"
                "--c-file"
                    "${TLD_DATA_C}"
                "${PROJECT_BINARY_DIR}/tlds.tld"
    WORKING_DIRECTORY ${CMAKE_SOURCE_DIR}
    MAIN_DEPENDENCY tld_parser
    DEPENDS ${TLD_FILES}
)
add_custom_target(${PROJECT_NAME} ALL DEPENDS ${TLD_DATA_C})
define_property(SOURCE
    PROPERTY GENERATED
    BRIEF_DOCS "The tld_data.c file is a table of all the TLDs defined in conf/tlds/... .ini files."
    FULL_DOCS "In the new version, the tld_data.c is always generated on the fly since it can be done with just C/C++."
)

Next I want to generate the tld dynamic and static libraries.

set(LIBTLD_SOURCES
    tld.cpp
    tld_compiler.cpp
    ${TLD_DATA_C}
    tld_domain_to_lowercase.c
    tld_emails.cpp
    tld_file.cpp
    tld_object.cpp
    tld_strings.c
)

##
## TLD library
##
project(tld)
configure_file(
    tld.h.in
    ${PROJECT_BINARY_DIR}/tld.h
)
add_library(${PROJECT_NAME} SHARED
    ${LIBTLD_SOURCES}
)
set_target_properties(${PROJECT_NAME} PROPERTIES
    VERSION ${LIBTLD_VERSION_MAJOR}.${LIBTLD_VERSION_MINOR}
    SOVERSION ${LIBTLD_VERSION_MAJOR}
)
install(
    TARGETS ${PROJECT_NAME}
    LIBRARY DESTINATION lib
    COMPONENT runtime
)
install(
    FILES ${PROJECT_BINARY_DIR}/tld.h
    DESTINATION include
    COMPONENT development
)

##
## TLD static library
##
project(tld_static)
add_library(${PROJECT_NAME} STATIC
    ${LIBTLD_SOURCES}
)
add_dependencies(${PROJECT_NAME}
    tld
)
# We need the -fPIC to use this library as extension of PHP, etc.
set_target_properties(tld_static PROPERTIES COMPILE_FLAGS -fPIC)

install(
    TARGETS ${PROJECT_NAME}
    ARCHIVE DESTINATION lib
    COMPONENT development
)

I added the add_dependencies() to the tld_static project to depend on tld thinking that would help my case, but I still see two run of the tld_parser...

I also had a test that directly includes the tld_data.c file (that way I can directly verify that the tld() function returns what I would expect from the exact source data).

Adding the add_dependencies() to the test worked as expected. Instead of getting the tld_data.c file created three times, now it's only twice.

project(tld_internal_test)
add_executable(${PROJECT_NAME}
    tld_internal_test.cpp
)
add_dependencies(${PROJECT_NAME}
    tld
)
add_test(
    NAME ${PROJECT_NAME}
    COMMAND ${PROJECT_NAME}
    WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
)

By killing the parallel build feature, it works as expected, however, there should be no good reasons for me to do that (see this cmake question for reference).

What I'm wondering, though, is:

How do you debug such an issue?

cmake's Makefile's are enormous, so reading those is quite a killer. If I knew what to search for, I suppose I could at least look at specific targets instead of trying to understand all that's happening in there (most of which has nothing to do with my issue).


¹ The code is in a branch at the moment. I'm hoping to have it all wrapped up soon at which point it will replace the main branch. The tld() function interface will not have changed very much. The implementation, however, will be quite different, more flexible, dynamically updatable.

Alexis Wilke
  • 19,179
  • 10
  • 84
  • 156
  • Instead of `add_custom_target(${PROJECT_NAME} ALL DEPENDS ${TLD_DATA_C})` try `add_library(.... OBJECT)` and linking with it. maybe try a static lbrary, but dunno how to include it in shared library. `cmake's Makefile'` Yes, so if you want to debug, you debug the build system. But anyway, move to Ninja - it's faster, and it can build `ninja -t browse` a dependency graph for you. `COMMAND "tld_parser"` use `$`. Are you 100% sure, that `tld_parser` does not change modification times of any other file? Could be it's opening any of `TLD_FILES` files for modification. – KamilCuk Dec 31 '21 at 23:55
  • https://github.com/m2osw/libtld/blob/SNAP-737-remove-qt-dependency/libtld/tld_compiler.cpp#L1402 -- here? It's opening read/write, am I right? Are modification times of `TLD_FILES` getting updated? – KamilCuk Jan 01 '22 at 00:02
  • 1
    It is `tld_data` target which creates the file `tld_data.c`. So you libraries should depends (via `add_dependencies`) from that target, not from the `tld` one. Note also to that description for [add_custom_target](https://cmake.org/cmake/help/latest/command/add_custom_command.html): Do not list the output in more than one independent target that may build in parallel or the two instances of the rule may conflict (instead use the `add_custom_target()` command to drive the command and make the other targets depend on that one). – Tsyvarev Jan 01 '22 at 00:05
  • @KamilCuk I verified with `stat ` and clearly it doesn't change the modification time (or any time if that matter). The default mode when opening an input stream is `ios_base::in`, (see [basic_istream](https://en.cppreference.com/w/cpp/io/basic_ifstream/basic_ifstream)) – Alexis Wilke Jan 01 '22 at 00:39
  • @KamilCuk Ah! Good catch! :-) Fixed. Thank you! – Alexis Wilke Jan 01 '22 at 00:49
  • @Tsyvarev It works using tld_data instead of tld (or as I tried before tld_data.c) in the add_dependencies() works! Now I only see one run of tld_parser and when I re-run make, it does not rebuild every time. Perfect! If you'd like to write an answer, I'll mark it as the valid answer. – Alexis Wilke Jan 01 '22 at 10:08

0 Answers0