Exclude manylinux wheels when downloading from pip

Question

At $company we run an internal pypi server to shield ourselves from public pypi downtime. We also build wheels to avoid installation overhead for binary packages. One common task is to import packages from public pypi which essentially boils down to:

pip install --download . --no-binary :all: $PACKAGE  # download a source distribution

and

pip wheel $PACKAGE  # build a binary distribution (or use a cached version that's already on our internal pypi)

Under the latest pip, this may download / install manylinux wheels during the pip wheel phase. Due to bundling .so files inside the wheel, these are incompatible with our security requirements at $company. How can we continue the same workflow while avoiding the manylinux wheels?

Currently we're downgrading to pip<8, but this seems non-ideal

Answer 1

Workaround

As a workaround you can create file _manylinux.py in current workdir, or in the site-packages with following content:

manylinux1_compatible = False

This will ensure that manylinux1 wheels won't be downloaded when you run your commands:

pip wheel -w path/to/wheeldir -f path/to/wheeldir -r requirements.txt

You don't really need to use two commands pip install --download and pip wheel, but you can use single command above. (please note, that pip install --download is deprecated in favor of pip download command).

Command as I suggested will:

1. Check if wheel already exists in path/to/wheeldir and will not touch that wheel.
2. If new version was detected in requirements file (or you can replace -r requirements.txt with package and exact version package==X.X.X) and wheel is universal -- it will download that wheel and store it into path/to/wheeldir.
3. If package requires compiling (example numpy, or matplotlib) -- source tgz will be downloaded and pip will build the wheel, and resulting wheel will be stored in the path/to/wheeldir.

I've tested this solution, which was [suggested in the pypa/pip#3689. It is not the solution we need, but it is workable and I use it.

Where can you put _manylinux.py file?

PIP attempts to import _manylinux module.

Q: Where Python will look for it?
A: Python follows the module search path.

So you can put your file in following locations:

1. current work dir.
2. Path in your PYTHONPATH.
3. Installation default path.

First two paths are simple, but how you find your installation default path? Simply run following command:

python -c "import os.path as p;print(p.dirname(p.realpath(p.__file__)))"

Hope, this helps.

Answer 2

For those looking for an installable workaround, I've created no-manylinux

The usage is relatively straightforward:

# First install no-manylinux
pip install no-manylinux
# Now subsequent invocations of pip will ignore manylinux wheels
pip install ...
# To restore the original behaviour, simply `pip uninstall no-manylinux`

It works by installing a _manylinux.py file with manylinux_compatible = False as the contents

Answer 3

You seem to want to use only source distributions and not wheels when using wheels. To do this add this --no-use-wheel to your pip command.

This sounds a bit counter intuitive to use this when building a wheel, but it really means Do not Find and prefer wheel archives when searching indexes and find-links locations. This should force the fetching of remote source dists instead of wheels during the wheel phase for any of the deps you depend on.