2

I am trying to build python wheels for a package (lap) for the aarch64 architecture. My host environment is WSL2 with Ubuntu 20.04 and docker. Target is Buildroot GNU/Linux. So no compiler is available on the target. My goal is to setup a cross-build environment for aarch64 using qemu. As described in Run a AArch64 native container on x86 with emulation we can use a containerized environment available to run on AArch64 to build wheels to the current specification with QEMU emulator. Steps I am doing:

  1. installing qemu packages in WSL2 sudo apt-get install qemu binfmt-support qemu-user-static
  2. registering scripts: docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
  3. Testing the emulation environment docker run --rm -t arm64v8/ubuntu uname -m and it returns aarch64 so, I believe the installation was successful, the emulation is working. Also, qemu-aarch64-static is available in /usr/bin/
  4. Now I clone the project lap (in WSL2) and cd lap/ and it contains setup.py but when I execute below command to build the wheels 
    docker run --rm -v `pwd`:/io quay.io/pypa/manylinux2014_aarch64 bash -c '/opt/python/cp38-cp38/bin/python ./setup.py bdist_wheel'

I get below error

```
WARNING: The requested image's platform (linux/arm64/v8) does not match the detected host platform (linux/amd64/v4) and no specific platform was requested
/opt/python/cp38-cp38/bin/python: can't open file './setup.py': [Errno 2] No such file or directory
```

Now I am not sure how to pass qemu-aarch64-static to above docker command?

Can any one please let me know how to resolve this and build python wheels using QEMU?

Thanks in advance.

P.S: Please let me know if any info is missing.

Preeti
  • 535
  • 1
  • 6
  • 30
  • Hi @matt, yes I have checked the container and `/opt/python/cp38-cp38/bin/python` is reference to the path in the container. – Preeti Aug 22 '23 at 11:16
  • 1
    Where did you get the upolygon program setup.py? I started the ubuntu container from your step 3. I installed python3, python3-venv, gcc, python3-dev, and git. Then I cloned to upolygon repository. I installed `wheel` into a virtual environment and I ran the `python setup.py bdist_wheel` command from the upolygon source code. – matt Aug 22 '23 at 11:26
  • How did you install `python3, python3-venv, gcc, python3-dev, and git` in the container? When I ran the bash shell on the container and logged in, there was no `apt` or any other package managers to install tools. So can you please let me know how did you install these tools in the container? – Preeti Aug 22 '23 at 11:33
  • 1
    In the ubuntu container, I used apt. I tried with the `quay.io/pypa/manylinux2014_aarch64` and only had to install `wheel` in a python environment becomes it comes with everything else. git and python were all installed ok. The manylinux one doesn't come with apt. I was able to use python3.8 and build upolygon without installing anything else. – matt Aug 22 '23 at 11:40
  • 1
    The problem you're having, You're not accessing "setup.py" it is not automatically inside of your container. This "-v `pwd`:/io" will attach the current directory to `io` You either need to cd into io or reference the setup.py accordingly. – matt Aug 22 '23 at 11:45
  • Okay, I tried to install them manylinux container quay.io/pypa/manylinux2014_aarch64. I will try to install them in arm64v8/ubuntu container. Can you please document the steps you have executed in the answer section? I believe this would be useful for others also... Thank you – Preeti Aug 22 '23 at 11:48

1 Answers1

1

All seem to be working fine, but the last step you're not accessing the actual file.

Since I don't know docker so well. First I start a aarch64 shell.

docker run -it quay.io/pypa/manylinux2014_aarch64 bash

[root@637db2c1af5e /]# uname -m
aarch64

Then from inside of the container, I just build the program like I would normally.

git clone https://github.com/gatagat/lap

Then install some dependencies.

python3.8 -m pip install numpy cython

Then I can build the wheel.

python3.8 setup.py bdist_wheel

Then I have a "dist" folder with.

-rw-r--r--. 1 root root 1.7M Aug 22 11:59 lap-0.5.dev0-cp38-cp38-linux_aarch64.whl

matt
  • 10,892
  • 3
  • 22
  • 34
  • Hi @matt, the python version on my embedded board is 3.10.4 and wheel is created using python3.8. So when I try to install the wheel, I get `.whl is not a supported wheel on this platform.` But then I renamed the wheel to `lap-0.5.dev0-cp310-cp310-linux_aarch64.whl` (cp310) and now its installed. I hope this is fine for now. or should I upgrade python to python3.10 in container `manylinux2014_aarch64` and then rebuild the wheel using `python3.10 setup.py bdist_wheel` ? – Preeti Aug 31 '23 at 13:23
  • 1
    @Preeti I think the container comes with a couple versions of python including 3.10. I would definitely try compiling with 3.10 otherwise you might have a strange error. – matt Aug 31 '23 at 13:28