5

I'm building my own install of Python, then using pip to install some packages into it. I want to use the pre-built binary wheels for packages like Cryptography.

  • Python 2.7.15 / 2.7.16
  • Pip 19.0.3
  • Setuptools 40.8.0

On GNU/Linux is just works: it grabs the manylinux1 wheel and everything works just fine.

On MacOS, it refuses to download any of the binary wheels for most versions. I've added lots of -v options to pip, but all it says is:

$ mypython -s -u -m pip install -v --only-binary 'cryptography' 'cryptography==2.6.1'
  ...
Skipping link https://files.pythonhosted.org/packages/.../cryptography-2.6.1-cp27-cp27m-macosx_10_6_intel.whl#sha256=... (from https://pypi.org/simple/cryptography/) (requires-python:>=2.7,!=3.0.*,!=3.1.*,!=3.2.*,!=3.3.*); it is not compatible with this Python
  ...
  Could not find a version that satisfies the requirement cryptography==2.6.1 (from versions: 1.0.1, 1.0.2, 1.1, 1.1.1, 1.1.2, 1.2, 1.2.1, 1.2.2, 1.3.1, 1.3.2, 1.3.3, 1.3.4, 1.4, 1.5, 1.5.1, 1.5.2, 1.5.3, 1.6, 1.7, 1.7.1, 1.8, 1.8.1, 1.8.2)

I've tried to understand why those particular versions are OK but not the latest, and the only thing I can see is that those versions have a specific x86_64 wheel package, while the later ones only have the fat binary intel wheel packages. My Python has a package definition of:

>>> import distutils.util
>>> distutils.util.get_platform()
'macosx-10.12-x86_64'

So I wondered if that was it. I modified my Python build to use MACOSX_DEPLOYMENT_TARGET=10.6 and added --enable-universalsdk=/ --with-universal-archs=intel to the configure line, and now my Python reports this:

>>> import distutils.util
>>> distutils.util.get_platform()
'macosx-10.6-intel'

However, I still get exactly the same messages from pip when I try to install.

So I'm wondering, is there any way I can get pip to be more informative and tell me exactly what about these binary packages it doesn't like, that is causing it to say "is not compatible" and skip them?

sinoroc
  • 18,409
  • 2
  • 39
  • 70
MadScientist
  • 92,819
  • 9
  • 109
  • 136
  • Also include in Q: output of `from wheel.pep425tags import get_supported; print(*get_supported())`. – wim Mar 28 '19 at 20:14
  • Updated with versions, sorry about that. I get `ImportError: No module named wheel.pep425tags` when I try to use that command. Note I'm on Python 2.7 if that matters. – MadScientist Mar 28 '19 at 20:26
  • 1
    Aha! I found `from setuptools import pep425tags` with a `get_supported()` method. – MadScientist Mar 28 '19 at 20:49
  • 2
    OK with that info I can see that I'm only accepting `cp27mu` and `none` as the second element of the tuple, while Cryptography only provides `cp27m`. So it looks like my Python is building with UCS4... sure enough, I find `--enable-unicode=ucs4` in my build scripts. Looks like it was inherited from Linux where that appears to be more common. Let me try without ucs2 instead. – MadScientist Mar 28 '19 at 21:10
  • That did it! Thanks! – MadScientist Mar 28 '19 at 21:37

2 Answers2

7

Thanks to a hint from @wim I found this command:

$ mypython
  ...
>>> from setuptools.pep425tags import get_supported
>>> for t in get_supported(): print(str(t))
...

This shows the full list of tuples used to match supported packages. Using this information I was able to compare it to the PyPI downloads and discover that I had built my MacOS Python with UCS4 support (which is common on Linux) where relatively few PyPI packages provide wide unicode binary wheels for MacOS.

I have no particular reason to prefer UCS4 so I switched my MacOS builds to UCS2 and it worked!

Hopefully this information will prove helpful to someone else.

Dirk Herrmann
  • 5,550
  • 1
  • 21
  • 47
MadScientist
  • 92,819
  • 9
  • 109
  • 136
6

Regarding the tags, recent versions of pip are able to output the full list of tags compatible with the currently running Python interpreter via the debug command:

$ path/to/pythonX.Y -m pip debug --verbose
WARNING: This command is only meant for debugging. Do not use this with automation for parsing and getting these details, since the output and options of this command may change without notice.
[...]
Compatible tags: 87
  cp38-cp38-manylinux2014_x86_64
  cp38-cp38-manylinux2010_x86_64
  cp38-cp38-manylinux1_x86_64
  cp38-cp38-linux_x86_64
  cp38-abi3-manylinux2014_x86_64
  cp38-abi3-manylinux2010_x86_64
  cp38-abi3-manylinux1_x86_64
  cp38-abi3-linux_x86_64
  cp38-none-manylinux2014_x86_64
  cp38-none-manylinux2010_x86_64
  cp38-none-manylinux1_x86_64
  cp38-none-linux_x86_64
  cp37-abi3-manylinux2014_x86_64
  cp37-abi3-manylinux2010_x86_64
  cp37-abi3-manylinux1_x86_64
  cp37-abi3-linux_x86_64
[...]
sinoroc
  • 18,409
  • 2
  • 39
  • 70