12

Consider the usual scenario - I want to create a virtual environment and install some packages. Say

python3 -m venv venv
source venv/bin/activate
pip install databricks-cli

During the installation, I get an error

Building wheels for collected packages: databricks-cli
Building wheel for databricks-cli (setup.py) ... error
ERROR: Command errored out with exit status 1:
command: /home/paulius/Documents/wheeltest/venv/bin/python3 -u -c 'import sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-m7jmyh1m/databricks-cli/setup.py'"'"'; __file__='"'"'/tmp/pip-install-m7jmyh1m/databricks-cli/setup.py'"'"';f=getattr(tokenize, '"'"'open'"'"', open)(__file__);code=f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' bdist_wheel -d /tmp/pip-wheel-maxix98x
   cwd: /tmp/pip-install-m7jmyh1m/databricks-cli/
Complete output (8 lines):
/tmp/pip-install-m7jmyh1m/databricks-cli/setup.py:24: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import imp
usage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
 or: setup.py --help [cmd1 cmd2 ...]
 or: setup.py --help-commands
 or: setup.py cmd --help

error: invalid command 'bdist_wheel'
----------------------------------------
ERROR: Failed building wheel for databricks-cli

While it is benign (the installation actually works), it is still annoying.

I know that pip install wheel resolves this, but wheel does not come with the virtual environment by default. So should I always add it to my requirements.txt, or maybe this is something that can be solved by the package maintainer (in this case databricks-cli) and hence I should open an issue in their Github?

Update: note that the wheel is not necessary to install wheels, in this example bunch of dependencies get successfully downloaded and installed as wheels. The only databricks-cli package gets the error, as it does not have a wheel, but for some reason, pip tries to build it.

Badhan Sen
  • 617
  • 1
  • 7
  • 18
psarka
  • 1,562
  • 1
  • 13
  • 25
  • Have you tried installing the [wheel-file](https://pypi.org/project/databricks-cli/#files) with `pip install databricks_cli-0.14.3-py2-none-any.whl`? With that pip doesn´t need to build a wheel from source. – Klim Bim Jul 14 '21 at 09:19
  • 2
    @KlimBim No, I have not. I expect pip to find the best file on it's own, without me searching for the approriate binary. (And I think it does that, in this case the wheel is for python2, so it is not suitable for my venv). – psarka Jul 14 '21 at 11:52
  • I am wondering why it isn´t working. If you install your downloaded `pip install databricks-cli-0.14.3.tar.gz` and `pip` doesn´t find `wheel`, it uses `setup.py` in the tarball. I tried it and `pip` used the legacy `setup.py install` – Klim Bim Jul 14 '21 at 12:39

2 Answers2

9

Update 3:

To prevent it from the maintainer's perspective use:

setup_requires=["wheel"]

it looks like you're on Linux and using the pre-installed or otherwise modified Python and its setuptools.

I've experienced the same thing due to Debian cutting off parts of the packages in not quite a sane way and I have mainly had issues with the prebuilt python-setuptools and likes. Check if the version matches and if not, install setuptools from pip, that might help in the future.

I have setuptools 45.2.0 and had no issues installing the package as a wheel. Then I uninstalled wheel and dropped the cache dir and it installed it properly even from the tar.gz source.

Update 2:

It might or might not be resolved by requiring wheel package. If the setup.py expects bdist_wheel to be present prior to the installation (most likely), adding it to the setup() function will not help and a manual check for the package within the setup script (+ perhaps a reference in README) are necessary so an end-user can install it properly.

For example if it's not present on the system, just print a warning and call an exit(). That would be the least a maintainer should do.

Update:

Yes, in the case where you encounter bdist_wheel command missing you need to install wheel with pip instal wheel.

It's not required, but it's recommended. Pip will work just fine without wheels, but you'll be installing from source (tar.gz, .zip or .egg).

See the packaging discussion for whether to use wheel or egg (or source).

Peter Badida
  • 11,310
  • 10
  • 44
  • 90
  • I still don't quite understand. If I look at the setup.py of the example databricks-cli file (neat and small here https://github.com/databricks/databricks-cli/blob/master/setup.py) I don't see anything indicating that a wheel should be built, so why is it attempted and how should the maintainers prevent it? – psarka Jul 14 '21 at 11:59
  • Also see the Update to the question I made to clarify your first initial answer – psarka Jul 14 '21 at 12:04
  • Thanks, but I think this is not correct, and this is in fact a pip bug, that was fixed -- with pip 21.1.3 (newest) I no longer get the error, but a nice warning instead. – psarka Jul 14 '21 at 12:25
  • @psarka Seems like quite a new bug. I have `20.3.3` on one of the machines and I have tested it even there - no issues. I'm glad it works after the update though. Perhaps try to verify with `pip install --no-cache-dir`. – Peter Badida Jul 14 '21 at 12:27
  • 1
    I get the error with `20.0.2`, so it must be somewhere in between. Thanks for the help! – psarka Jul 14 '21 at 12:30
4

This was a pip bug, and the solution is to upgrade the pip. With the newest version things look fine:

(venv) paulius@xps:~/Documents/wheeltest$ pip install databricks-cli
Collecting databricks-cli
  Using cached databricks-cli-0.14.3.tar.gz (54 kB)
Collecting click>=6.7
  Using cached click-8.0.1-py3-none-any.whl (97 kB)
Collecting requests>=2.17.3
  Using cached requests-2.26.0-py2.py3-none-any.whl (62 kB)
Collecting six>=1.10.0
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting tabulate>=0.7.7
  Using cached tabulate-0.8.9-py3-none-any.whl (25 kB)
Collecting idna<4,>=2.5
  Using cached idna-3.2-py3-none-any.whl (59 kB)
Collecting certifi>=2017.4.17
  Using cached certifi-2021.5.30-py2.py3-none-any.whl (145 kB)
Collecting urllib3<1.27,>=1.21.1
  Using cached urllib3-1.26.6-py2.py3-none-any.whl (138 kB)
Collecting charset-normalizer~=2.0.0
  Using cached charset_normalizer-2.0.1-py3-none-any.whl (35 kB)
Using legacy 'setup.py install' for databricks-cli, since package 'wheel' is not installed.
Installing collected packages: urllib3, idna, charset-normalizer, certifi, tabulate, six, requests, click, databricks-cli
    Running setup.py install for databricks-cli ... done
Successfully installed certifi-2021.5.30 charset-normalizer-2.0.1 click-8.0.1 databricks-cli-0.14.3 idna-3.2 requests-2.26.0 six-1.16.0 tabulate-0.8.9 urllib3-1.26.6

Note the Using legacy 'setup.py install' ... line.

This is a related issue in the pip github https://github.com/pypa/pip/issues/8302. Not exactly that, but there is an explanation in the comments on what's the wheel building logic supposed to by.

psarka
  • 1,562
  • 1
  • 13
  • 25