11

I have a mainly c++ project that I use CMake to manage. After setting cmake_install_prefix and configuring, it generates makefiles which can then be used to build and install with the very standard:

make
make install

At this point, my binaries end up in cmake_install_prefix, and they can be executed with no additional work. Recently I've added some Python scripts to a few places in the source tree, and some of them depend on others. I can use CMake to copy the Python files+directory structure to the cmake_install_prefix, but if I go into that path and try to use one of the scripts, Python cannot find the other scripts used as imports because PYTHONPATH does not contain cmake_install_prefix. I know you can set an environment variable with CMake, but it doesn't persist across shells, so it's not really "setup" for the user for more than the current terminal session.

The solution seems to be to add a step to your software build instructions that says "set your PYTHONPATH". Is there any way to avoid this? Is this the standard practice for "installing" Python scripts as part of a bigger project? It seems to really complicate things like setting up continuous integration for the project, as something like Jenkins has to be manually configured to inject environment variables, whereas nothing special was required for it to build and execute executables built from c++ code.

boardrider
  • 5,882
  • 7
  • 49
  • 86
David Doria
  • 9,873
  • 17
  • 85
  • 147

1 Answers1

10

Python provides sys.path list, which is used for search modules with import directives. You may adjust this list before include your modules:

script1.py:

# Do some things useful for other scripts

script2.py.in:

# Uses script1.py.
...
sys.path.insert(1, "@SCRIPT1_INSTALL_PATH@")
import script1
...

CMakeLists.txt:

...
# Installation path for script1. Depends from CMAKE_INSTALL_PREFIX.
set(SCRIPT1_INSTALL_PATH ${CMAKE_INSTALL_PREFIX}/<...>)

install(FILES script1.py DESTINATION ${SCRIPT1_INSTALL_PATH}

# Configure 'sys.path' in script2.py, so it may find script1.py.
configure_file("script2.py.in" "script2.py" @ONLY)

set(SCRIPT2_INSTALL_PATH ${CMAKE_INSTALL_PREFIX}/<...>)
install(FILES script2.py DESTINATION ${SCRIPT2_INSTALL_PATH}
...

If you want script2.py to work both in build tree and in install tree, you need to have two instances of it, one which works in build tree, and one which works after being installed. Both instances may be configured from single .in file.

In case of compiled executables and libraries, similar mechanism is uses for help binaries to find libraries in non-standard locations. It is known as RPATH.

Because CMake

  • knows every binary created (it tracks add_executable and add_library calls),

  • knows linkage between binaries (target_link_libraries call is also tracked),

  • has full control over linking procedure,

CMake is able to automatically adjust RPATH when install binaries.

In case of Python scripts CMake doesn't have such information, so adjusting linkage path should be performed manually.

Tsyvarev
  • 60,011
  • 17
  • 110
  • 153
  • So it seems this will produce a working "installed" version. Is there anyway to have, in addition, a more immediate/intermediate way to run the scripts for development. That is, change a .py file, run that whole CMake procedure, then try/test it seems a bit clunky. Is there anyway to make the scripts work "in-place" as well (from the source tree, for example)? – David Doria Jan 21 '17 at 12:45
  • `Is there anyway to make the scripts work "in-place" as well (from the source tree, for example)?` In the scripts you may use some sort of annotations, like `sys.path.append(/path/to/scriptA) # $$scriptA$$`. When want to install a script, firstly find all `$$...$$` lines in it and replace them with appropriate expressions, then install resulted file. In any case **installed** script should be *different* from one **in source tree** (unless your want to adjust environment or script's parameters). – Tsyvarev Jan 21 '17 at 14:45
  • Use `install(PROGRAMS script1.py...)` if you want the script to be executable. [`PROGRAMS`](https://cmake.org/cmake/help/v3.10/command/install.html#installing-files) does this. – phoenix Mar 16 '18 at 13:01