Is there an easy way of skip download_step if folder exists, in cmake ExternalProject_Add

Question

I need the ExternalProject_Add in CMakeLists.txt to skip download_step for git downloads if there is already a folder with the repo cloned, like the update_step if you provide a GIT_TAG, but don't want to write a custom DOWNLOAD_COMMAND. Basically just need the project to build fully offline if the dependencies where added manually to the folder project.

For example:

ExternalProject_Add(civ
  SOURCE_DIR            ${PROJECT_SOURCE_DIR}/test
  BUILD_COMMAND         ${MAKE_EXE}
  BINARY_DIR            "./bin"
  INSTALL_COMMAND       ""
  GIT_REPOSITORY        "https://github.com/test/test"
  GIT_TAG               "v1.0"
  LOG_DOWNLOAD          on
  CMAKE_ARGS
    "-DTuberosumTools_DIR=${TUBEROSUMTOOLS_DIR}"
)

this, deletes test folder and clones it again every time the project is builded for the first time(if i manually delete build folder), i can build after that totally offline, but really need to use it offline always. Already tried to set a cache variable like:

set(FRESH_DOWNLOAD off CACHE BOOL "download a fresh copy of all dependencies")

include(ExternalProject)
ExternalProject_Add(civ
  SOURCE_DIR            ${PROJECT_SOURCE_DIR}/test
  BUILD_COMMAND         ${MAKE_EXE}
  BINARY_DIR            "./bin"
  INSTALL_COMMAND       ""
  GIT_REPOSITORY        "https://github.com/test/test"
  GIT_TAG               "v1.0"
  if(NOT FRESH_DOWNLOAD)
    DOWNLOAD_COMMAND      ""
  endif()
  CMAKE_ARGS
    "-DTuberosumTools_DIR=${TUBEROSUMTOOLS_DIR}"
)

to completly disable download unless indicated, but the if inside the call to ExternalProject_Add(), obviusly does not work, and using the if outside introduces extra code a little harder to mantain, and its ugly.

Any simple alternative is valid, thanks in advance.

for better understanding of the correct answer, passing empty string is not that easy, my temporary solution was set the `DOWNLOAD_COMMAND` to `"dir"`, i would not recommend it but that way works both in unix and windows, for unix alone could have been `"true"` instead. — Darian Lopez Utra, Dec 07 '20 at 17:04

score 1 · Accepted Answer · answered Dec 06 '20 at 09:32

When call a function or macro, CMake allows to substitute a variable, which contains several parameters at once. That is, you may conditionally define a variable which is responsible for DOWNLOAD step:

set(FRESH_DOWNLOAD off CACHE BOOL "download a fresh copy of all dependencies")

include(ExternalProject)
if (NOT FRESH_DOWNLOAD)
    # Define the variable to disable DOWNLOAD step
    set(civ_DOWNLOAD DOWNLOAD_COMMAND      "")
else()
    # Define an empty variable.
    # This actually can be omitted since absent variable is treated as empty.
    set(civ_DOWNLOAD)
endif()

ExternalProject_Add(civ
  SOURCE_DIR            ${PROJECT_SOURCE_DIR}/test
  BUILD_COMMAND         ${MAKE_EXE}
  BINARY_DIR            "./bin"
  INSTALL_COMMAND       ""
  GIT_REPOSITORY        "https://github.com/test/test"
  GIT_TAG               "v1.0"
  ${civ_DOWNLOAD} # IMPORTANT: Do not use double quotes here.
  CMAKE_ARGS
    "-DTuberosumTools_DIR=${TUBEROSUMTOOLS_DIR}"
)

actually, `${civ_DOWNLOAD}` does not seen to been converted to `DOWNLOAD_COMMAND ""`, i copy/paste your code — Darian Lopez Utra, Dec 06 '20 at 16:59
Yes, passing empty parameter ("") indirectly could be non-trivial... I will investigate this further. — Tsyvarev, Dec 06 '20 at 17:11

score 1 · Answer 2 · answered Nov 09 '21 at 13:07

1

You can try FetchContent and then ExternalProject with directory downloaded by FetchContent. Something like that:

include(FetchContent)
FetchContent_Declare(
    mylib
    GIT_REPOSITORY "git@github.com:username/mylib.git"
    GIT_TAG <some sha>
)
FetchContent_MakeAvailable(mylib)

include(ExternalProject)
ExternalProject_Add(
    mylib
    SOURCE_DIR "_deps/mylib-src/"
    CONFIGURE_COMMAND ""
    INSTALL_COMMAND ""
    BUILD_COMMAND "pwd"
)

answered Nov 09 '21 at 13:07

E. Nerush

69
7

This is kind of a hacky workaround in my opinion, but I get your point, the correct answer should remain the same though. In fact recently I started to use something like this in my code, thanks for the answer anyways. – Darian Lopez Utra Nov 18 '21 at 05:17

score 0 · Answer 3 · answered Dec 06 '20 at 09:29

Update/Patch Step Options: Whenever CMake is re-run, by default the external project’s sources will be updated if the download method supports updates (e.g. a git repository would be checked if the GIT_TAG does not refer to a specific commit).

UPDATE_DISCONNECTED <bool> When enabled, this option causes the update step to be skipped. It does not, however, prevent the download step. The update step can still be added as a step target (see ExternalProject_Add_StepTargets()) and called manually. This is useful if you want to allow developers to build the project when disconnected from the network (the network may still be needed for the download step though).

ref: https://cmake.org/cmake/help/latest/module/ExternalProject.html

this answers nothing, i already read the docs, sorry did not say that, but thought it was implied — Darian Lopez Utra, Dec 06 '20 at 16:26

Is there an easy way of skip download_step if folder exists, in cmake ExternalProject_Add

3 Answers3