0

Note: There are several similar questions, which I have seen and read. None of them are the precise problem I'm having, and none of their answers work for me.

I have installed several Python versions (Python 2.7, 3.8, 3.9, and 3.10) on my macOS Ventura system using Pyenv. The installed directories include a python-config file in /bin and an /include directory with many header files, so I do not need to install python-devel or similar:

$ ls -al /Users/williamsn/.pyenv/versions/3.10.9/bin/*-config
lrwxr-xr-x  1 williamsn  staff    17 Dec 19 17:24 
/Users/williamsn/.pyenv/versions/3.10.9/bin/python-config -> python3.10-config
lrwxr-xr-x  1 williamsn  staff    17 Dec 19 17:24 
/Users/williamsn/.pyenv/versions/3.10.9/bin/python3-config -> python3.10-config
-rwxr-xr-x  1 williamsn  staff  2073 Dec 19 17:24 
/Users/williamsn/.pyenv/versions/3.10.9/bin/python3.10-config
$ ls -al /Users/williamsn/.pyenv/versions/3.10.9/include/python3.10/
total 1192
drwxr-xr-x  86 williamsn  staff   2752 Dec 19 17:24 .
drwxr-xr-x   3 williamsn  staff     96 Dec 19 17:24 ..
-rw-r--r--   1 williamsn  staff   3224 Dec 19 17:24 Python.h
...
-rw-r--r--   1 williamsn  staff   3026 Dec 19 17:24 import.h
...
-rw-r--r--   1 williamsn  staff  48878 Dec 19 17:24 pyconfig.h
...
-rw-r--r--   1 williamsn  staff   2863 Dec 19 17:24 weakrefobject.h

As a best practice, I work exclusively in virtualenvs to keep the base Python installation clean and pristine. I create my virtualenvs using the venv module built in to Python 3:

$ python3.10 -m venv my_venv

After doing so, there are two problems:

  1. The my_venv/include directory is empty (does not contain the python3.10 directory or any of the header files from the parent Python and does not symlink to the parent Python)
  2. There is no python-config / python3-config / python3.10-config in my_venv/bin.

So, for things requiring the Python headers:

  • If it tries to find the headers using python-config (which many projects do), it will fail.
  • If it tries to find the headers using the Python prefix and adding /include to it (which many projects do, notably Boost), it will fail.
  • If it tries to find the headers by importing sysconfig and calling sysconfig.get_paths(), it will succeed, albeit with paths outside the virtualenv.

Now, on a case-by-case basis, I can work around this, and already have. I can manually copy over python_config and then either export CPATH to add /Users/williamsn/.pyenv/versions/3.10.9/include/python3.10 or modify the virtualenv to symlink to /Users/williamsn/.pyenv/versions/3.10.9/include. These work, but they don't seem right to me. I can't make a global workaround anywhere (such as exporting CPATH in my Bash profile), because I'm working with multiple Python versions, and I'd end up with the wrong headers half the time. It seems to me like a virtualenv that has an empty include directory and is missing python-config when its parent has a full include directory and contains python-config is a broken virtualenv.

Is there an option I'm missing to include these pieces? Is this a bug I need to file against Python/venv?

oguz ismail
  • 1
  • 16
  • 47
  • 69
Nick Williams
  • 2,864
  • 5
  • 29
  • 43

0 Answers0