12

My primary development machine is x86_64 while some of my deploy environments are arm7vl (Raspberry Pi). For most Python development, this isn't a problem, but some Python libraries are are only available exclusively in PyPI for x86_64 or piwheels for aarmv7l. This has lead to some difficulty using Poetry. As a simple example, here's a pyproject.toml file created on the x86_64 machine:

[tool.poetry]
name = "poetrytest"
version = "0.1.0"
description = ""
authors = ["Your Name <you@example.com>"]

[tool.poetry.dependencies]
python = "^3.9"
opencv-python = "^4.5.5"

[tool.poetry.dev-dependencies]

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

The corresponding poetry.lock file contains hashes for files pulled from PyPI and when you run poetry install everything works as expected. However, if you copy these two files over to a Raspberry Pi, the install fails to find an appropriate .whl file and therefore falls back to trying to build from source which takes roughly 2 hours and fails :-(

To make it work on the Pi, you need to add this block to pyproject.toml:

[[tool.poetry.source]]
name = "piwheels"
url = "https://www.piwheels.org/simple/"

...then delete poetry.lock and run poetry install. This will re-generate the lock file, (now with entries from piwheels.org) and install everything as expected. However this isn't terribly useful, since it means that I can't version pyproject.toml or poetry.lock. I also can't include the above source snippet in the original pyproject.toml file, or the build on the x86_64 machine dies with Unable to find installation candidates.

So far, the only cross-platform way I can find to make this work is to keep everything versioned from the x86_64 machine and just run this on the Pi when I want to install something:

$ poetry export --without-hashes > requirements.txt
$ pip install --requirement requirements.txt

which... sucks. Surely, there must be a better way?

Daniel Quinn
  • 6,010
  • 6
  • 38
  • 61
  • I guess this is not possible at the moment: https://github.com/python-poetry/poetry/issues/4854 – finswimmer Jan 03 '22 at 12:31
  • Ah! Well that's a pity. At least I know that this is a known problem :-( – Daniel Quinn Jan 03 '22 at 14:08
  • Documentation here `https://python-poetry.org/docs/repositories/` states that you should be able to run `poetry config repositories.piwheels https://www.piwheels.org/simple/`. But ofc it doesn't fix your (our) problem – Emiliano May 14 '22 at 20:34

1 Answers1

0

A way around your issue could be to use conda instead of poetry. Then set the environment variable CONDA_SUBDIR=<your_platform> which forces the platform to use when building the virtual environment with:

conda env create ...

(Not tested) You should then be able to transfer the created virtual environment and use it in your deploy environment.

Greg7000
  • 297
  • 3
  • 15