Note: There are several similar questions, which I have seen and read. None of them are the precise problem I'm having, and none of their answers work for me.
I have installed several Python versions (Python 2.7, 3.8, 3.9, and 3.10) on my macOS Ventura system using Pyenv. The installed directories include a python-config
file in /bin
and an /include
directory with many header files, so I do not need to install python-devel or similar:
$ ls -al /Users/williamsn/.pyenv/versions/3.10.9/bin/*-config
lrwxr-xr-x 1 williamsn staff 17 Dec 19 17:24
/Users/williamsn/.pyenv/versions/3.10.9/bin/python-config -> python3.10-config
lrwxr-xr-x 1 williamsn staff 17 Dec 19 17:24
/Users/williamsn/.pyenv/versions/3.10.9/bin/python3-config -> python3.10-config
-rwxr-xr-x 1 williamsn staff 2073 Dec 19 17:24
/Users/williamsn/.pyenv/versions/3.10.9/bin/python3.10-config
$ ls -al /Users/williamsn/.pyenv/versions/3.10.9/include/python3.10/
total 1192
drwxr-xr-x 86 williamsn staff 2752 Dec 19 17:24 .
drwxr-xr-x 3 williamsn staff 96 Dec 19 17:24 ..
-rw-r--r-- 1 williamsn staff 3224 Dec 19 17:24 Python.h
...
-rw-r--r-- 1 williamsn staff 3026 Dec 19 17:24 import.h
...
-rw-r--r-- 1 williamsn staff 48878 Dec 19 17:24 pyconfig.h
...
-rw-r--r-- 1 williamsn staff 2863 Dec 19 17:24 weakrefobject.h
As a best practice, I work exclusively in virtualenvs to keep the base Python installation clean and pristine. I create my virtualenvs using the venv
module built in to Python 3:
$ python3.10 -m venv my_venv
After doing so, there are two problems:
- The
my_venv/include
directory is empty (does not contain thepython3.10
directory or any of the header files from the parent Python and does not symlink to the parent Python) - There is no
python-config
/python3-config
/python3.10-config
inmy_venv/bin
.
So, for things requiring the Python headers:
- If it tries to find the headers using
python-config
(which many projects do), it will fail. - If it tries to find the headers using the Python prefix and adding
/include
to it (which many projects do, notably Boost), it will fail. - If it tries to find the headers by importing
sysconfig
and callingsysconfig.get_paths()
, it will succeed, albeit with paths outside the virtualenv.
Now, on a case-by-case basis, I can work around this, and already have. I can manually copy over python_config
and then either export CPATH
to add /Users/williamsn/.pyenv/versions/3.10.9/include/python3.10
or modify the virtualenv to symlink to /Users/williamsn/.pyenv/versions/3.10.9/include
. These work, but they don't seem right to me. I can't make a global workaround anywhere (such as exporting CPATH
in my Bash profile), because I'm working with multiple Python versions, and I'd end up with the wrong headers half the time. It seems to me like a virtualenv that has an empty include
directory and is missing python-config
when its parent has a full include
directory and contains python-config
is a broken virtualenv.
Is there an option I'm missing to include these pieces? Is this a bug I need to file against Python/venv
?