20

Starting with python3.8, I noticed that the wheel package seems to be required to pip install packages without errors (at least they look like errors, but behave like warnings).

Note: in all of my examples, I am cleaning up in between by deactivating my environment, removing the environment, and clearing my pip cache:

deactivate
rm test_env/ -rf
rm ~/.cache/pip/ -rf

Example 1: The problem

python3.8 -m venv test_env
source test_env/bin/activate
pip install markuppy

I get the following output:

Collecting markuppy
  Downloading MarkupPy-1.14.tar.gz (6.8 kB)
Building wheels for collected packages: markuppy
  Building wheel for markuppy (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/k/test_env/bin/python3.8 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-tfm9bgxv/markuppy/setup.py'"'"'; __file__='"'"'/tmp/pip-install-tfm9bgxv/markuppy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-cpx9gxcn
       cwd: /tmp/pip-install-tfm9bgxv/markuppy/
  Complete output (6 lines):
  usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: setup.py --help [cmd1 cmd2 ...]
     or: setup.py --help-commands
     or: setup.py cmd --help
  
  error: invalid command 'bdist_wheel'
  ----------------------------------------
  ERROR: Failed building wheel for markuppy
  Running setup.py clean for markuppy
Failed to build markuppy
Installing collected packages: markuppy
    Running setup.py install for markuppy ... done
Successfully installed markuppy-1.14

It errored because there is no wheel package installed. This has been addressed in other posts on SO.

Example 2: Fixing the problem by installing wheel into the venv

If I install the wheel package directly after creating a new venv, this does not happen:

python3.8 -m venv test_env
source test_env/bin/activate
pip install wheel
pip install markuppy

Output:

Collecting markuppy
  Using cached MarkupPy-1.14.tar.gz (6.8 kB)
Building wheels for collected packages: markuppy
  Building wheel for markuppy (setup.py) ... done
  Created wheel for markuppy: filename=MarkupPy-1.14-py3-none-any.whl size=7413 sha256=52b3e5c3e317ae21724acd871fe3deb85dde9df305b20d16f2c5592c43b11e91
  Stored in directory: /home/k/.cache/pip/wheels/95/13/60/31c9d5f4cd012e491aeac154ef8b0ec964916523623eb02f0b
Successfully built markuppy
Installing collected packages: markuppy
Successfully installed markuppy-1.14

This also works given a requirements.txt file.

requirements.txt:

markuppy

python3.8 -m venv test_env
source test_env/bin/activate
pip install wheel
pip install -r requirements.txt

Output:

Collecting markuppy
  Using cached MarkupPy-1.14.tar.gz (6.8 kB)
Building wheels for collected packages: markuppy
  Building wheel for markuppy (setup.py) ... done
  Created wheel for markuppy: filename=MarkupPy-1.14-py3-none-any.whl size=7413 sha256=52b3e5c3e317ae21724acd871fe3deb85dde9df305b20d16f2c5592c43b11e91
  Stored in directory: /home/k/.cache/pip/wheels/95/13/60/31c9d5f4cd012e491aeac154ef8b0ec964916523623eb02f0b
Successfully built markuppy
Installing collected packages: markuppy
Successfully installed markuppy-1.14

Example 3: Adding the wheel package to requirements.txt does not help

It's important to know that if you place the wheel package into the requirements.txt file, and do not install it separately, you get the same problem as in Example 1:

requirements.txt:

wheel
markuppy

python3.8 -m venv test_env
source test_env/bin/activate
pip install -r requirements.txt

Output:

Collecting wheel
  Downloading wheel-0.37.0-py2.py3-none-any.whl (35 kB)
Collecting markuppy
  Downloading MarkupPy-1.14.tar.gz (6.8 kB)
Building wheels for collected packages: markuppy
  Building wheel for markuppy (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/k/test_env/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-z7cqzaej/markuppy/setup.py'"'"'; __file__='"'"'/tmp/pip-install-z7cqzaej/markuppy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-kw3_8ayt
       cwd: /tmp/pip-install-z7cqzaej/markuppy/
  Complete output (6 lines):
  usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: setup.py --help [cmd1 cmd2 ...]
     or: setup.py --help-commands
     or: setup.py cmd --help
  
  error: invalid command 'bdist_wheel'
  ----------------------------------------
  ERROR: Failed building wheel for markuppy
  Running setup.py clean for markuppy
Failed to build markuppy
Installing collected packages: wheel, markuppy
    Running setup.py install for markuppy ... done
Successfully installed markuppy-1.14 wheel-0.37.0

Wheel installed properly, but was not there in time to install the markuppy package.

Example 4: Installing wheel at the system level does not help

If I install the wheel package directly into the system itself, the venv that was created does not have access to it, so I get the same result as in Example 1.

sudo apt install python3-wheel
python3.8 -m venv test_env
source test_env/bin/activate
pip install markuppy

Output:

Collecting markuppy
  Downloading MarkupPy-1.14.tar.gz (6.8 kB)
Building wheels for collected packages: markuppy
  Building wheel for markuppy (setup.py) ... error
  ERROR: Command errored out with exit status 1:
   command: /home/k/test_env/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-5xkssq1l/markuppy/setup.py'"'"'; __file__='"'"'/tmp/pip-install-5xkssq1l/markuppy/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-aunof5xf
       cwd: /tmp/pip-install-5xkssq1l/markuppy/
  Complete output (6 lines):
  usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
     or: setup.py --help [cmd1 cmd2 ...]
     or: setup.py --help-commands
     or: setup.py cmd --help
  
  error: invalid command 'bdist_wheel'
  ----------------------------------------
  ERROR: Failed building wheel for markuppy
  Running setup.py clean for markuppy
Failed to build markuppy
Installing collected packages: markuppy
    Running setup.py install for markuppy ... done
Successfully installed markuppy-1.14

Example 5: Installing wheel at the system level and granting venv access to system site packages works

If I install wheel at the system level, then grant my venv access to system site packages (note the flag on the venv command) when I create it, then this issue does not happen.

sudo apt install python3-wheel
python3.8 -m venv test_env --system-site-packages
source test_env/bin/activate
pip install markuppy

Output:

Collecting markuppy
  Downloading MarkupPy-1.14.tar.gz (6.8 kB)
Building wheels for collected packages: markuppy
  Building wheel for markuppy (setup.py) ... done
  Created wheel for markuppy: filename=MarkupPy-1.14-py3-none-any.whl size=7414 sha256=cefe8d9f20cecaf72253cab1e18acbdcb6d30827d7a3fd5a74bbef4935bc2e44
  Stored in directory: /home/k/.cache/pip/wheels/95/13/60/31c9d5f4cd012e491aeac154ef8b0ec964916523623eb02f0b
Successfully built markuppy
Installing collected packages: markuppy
Successfully installed markuppy-1.14

The obvious downside to this approach is that it breaks isolation between your project's venv and the python packages installed on the system itself.

Example 6: None of this happens in Python3.7

python3.7 -m venv test_env
source test_env/bin/activate
pip install -U pip
pip install markuppy

Output:

Collecting markuppy
  Downloading MarkupPy-1.14.tar.gz (6.8 kB)
Using legacy 'setup.py install' for markuppy, since package 'wheel' is not installed.
Installing collected packages: markuppy
    Running setup.py install for markuppy ... done
Successfully installed markuppy-1.14

Although it does provide a warning that pip used setup.py install in lieu of wheel.

The Question

How are we supposed to deal with this wheel package? It seems that it's expected that we all have this package installed in most (all?) environments, but the environment tooling in the standard library does not automatically provide it.

I like the idea of keeping my project environments completely isolated from the system python environment, so it looks like the only options that I can see are:

  1. Manually install the wheel package immediately after creating a new venv.
  2. Using another tool besides venv to manage environments.
kbuilds
  • 991
  • 11
  • 19
  • 1
    Isn't this all crazy? If I got it right, when you install from `pypi` you don't need the `wheel` package, but you need it if for example if you clone or install with `pip install git+`. Shouldn't a venv include wheel by default? What would be the problem with that? – Francesco Feb 19 '22 at 13:25
  • @Francesco I think whether or not you have to build a wheel or download it directly from pypi depends on what operating system you are using, and if a wheel exists in pypi for that operating system/version. I might be wrong. I understand just enough about Python to build sweet apps. This system-level stuff baffles me sometimes. – kbuilds Mar 14 '22 at 17:13
  • 4
    I logged in just to say that 1) your post is awesome, super detailed 2) I wish that I had an answer for you, since I encountered this exact same issue just now. I am using a freshly created Ubuntu 20.04 VM, whose system Python is 3.8. Altho that system Python is Ubunutu's usual bare bones version (e.g. lacks pip, venv), it does have wheel already installed. And yet when I used venv to create a new virtual environment, and activated that new one, and then tried `pip install -r requirements.txt`, I got the same bdist_wheel error as you. Likewise, my cure was to first do `pip install wheel` – HaroldFinch Jun 03 '22 at 19:09
  • 1
    Maybe we should add an arg: `python -m venv venvdir -w` – MatrixManAtYrService Jul 15 '22 at 00:02

1 Answers1

7

When setting up a venv, before running anything else, just run pip install --upgrade pip wheel first. This was a good practice anyhow to insure you're using the latest version of pip to resolve your dependencies, hence the warning:

WARNING: You are using pip version 22.0.4; however, version 22.2.2 is available.
You should consider upgrading via the '/home/user/tmp/test_env/bin/python3 -m pip install --upgrade pip' command.

It does seem wheel is no longer included by default, as I was able to replicate on 3.10 as well. Specifying --system-site-packages to use system wheel would be expected, the whole point of venv is to avoid touching the system python. Not being used during a requirements.txt install also makes sense, as it's unlikely any of the packages have wheel listed as a dependency, so it's probably being installed in parallel.

Jason Kurzik
  • 71
  • 1
  • 4