Pytest core dump on Unbutu 22.04 LTS when using pyqt5 with mayavi and pyvirtualdisplay

Question

Want I want to do:

I want to have these requirements running on Ubuntu 22.04 LTS:

# Bug on Ub.22.04 but avoids AttributeError: 'Timer' object has no attribute 'start' on other OS
pyqt5==5.15.7

# These together work:
mayavi==4.7.4
PyVirtualDisplay==3.0
pytest==7.1.2

So I can run this test:

import pytest

from pyvirtualdisplay import Display
import os
display = Display(visible=0, size=(1280, 1024))
display.start()

from mayavi import mlab

def foo(tmp_path):

    assert 0 == 0;

Problem

Adding the PyQt5 to the list of dependencies seems to break the behavior of the program on Ubuntu 22.04 LTS (although it works on Ubuntu 20.04 LTS and MacOS for multiple version of Python 3.

The output error is the following:

Run python3 -m pip install pytest
  python3 -m pip install pytest
  cd tests
  python3 -m pytest
  shell: /usr/bin/bash --noprofile --norc -e -o pipefail {0}
Defaulting to user installation because normal site-packages is not writeable
Collecting pytest
  Downloading pytest-7.1.2-py3-none-any.whl (297 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 297.0/297.0 KB 8.8 MB/s eta 0:00:00
Collecting pluggy<2.0,>=0.12
  Downloading pluggy-1.0.0-py2.py3-none-any.whl (13 kB)
Collecting iniconfig
  Downloading iniconfig-1.1.1-py2.py3-none-any.whl (5.0 kB)
Requirement already satisfied: attrs>=19.2.0 in /usr/lib/python3/dist-packages (from pytest) (21.2.0)
Requirement already satisfied: packaging in /usr/local/lib/python3.10/dist-packages (from pytest) (21.3)
Collecting py>=1.8.2
  Downloading py-1.11.0-py2.py3-none-any.whl (98 kB)
     ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 98.7/98.7 KB 23.2 MB/s eta 0:00:00
Collecting tomli>=1.0.0
  Downloading tomli-2.0.1-py3-none-any.whl (12 kB)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/lib/python3/dist-packages (from packaging->pytest) (2.4.7)
Installing collected packages: iniconfig, tomli, py, pluggy, pytest
Successfully installed iniconfig-1.1.1 pluggy-1.0.0 py-1.11.0 pytest-7.1.2 tomli-2.0.1
Fatal Python error: Aborted
Current thread 0x00007f99b32c8000 (most recent call first):
  File "/home/runner/.local/lib/python3.10/site-packages/pyface/ui/qt4/init.py", line 36 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 883 in exec_module
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "<frozen importlib._bootstrap>", line 1050 in _gcd_import
  File "/usr/lib/python3.10/importlib/__init__.py", line 126 in import_module
  File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 171 in load
  File "/home/runner/.local/lib/python3.10/site-packages/pyface/base_toolkit.py", line 218 in import_toolkit
  File "/home/runner/.local/lib/python3.10/site-packages/pyface/base_toolkit.py", line 263 in find_toolkit
  File "/home/runner/.local/lib/python3.10/site-packages/pyface/toolkit.py", line 23 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 883 in exec_module
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "/home/runner/.local/lib/python3.10/site-packages/traitsui/qt4/toolkit.py", line 33 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 883 in exec_module
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1078 in _handle_fromlist
  File "/home/runner/.local/lib/python3.10/site-packages/traitsui/qt4/__init__.py", line 35 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 883 in exec_module
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "<frozen importlib._bootstrap>", line 1050 in _gcd_import
  File "/usr/lib/python3.10/importlib/__init__.py", line 126 in import_module
  File "/usr/lib/python3.10/importlib/metadata/__init__.py", line 171 in load
  File "/home/runner/.local/lib/python3.10/site-packages/pyface/base_toolkit.py", line 282 in find_toolkit
  File "/home/runner/.local/lib/python3.10/site-packages/traitsui/toolkit.py", line 110 in toolkit
  File "/home/runner/.local/lib/python3.10/site-packages/traitsui/toolkit_traits.py", line 43 in ColorTrait
  File "/home/runner/.local/lib/python3.10/site-packages/traits/trait_factory.py", line 40 in __call__
  File "/home/runner/.local/lib/python3.10/site-packages/traitsui/editors/code_editor.py", line 32 in CodeEditor
  File "/home/runner/.local/lib/python3.10/site-packages/traitsui/editors/code_editor.py", line 21 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 883 in exec_module
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "/home/runner/.local/lib/python3.10/site-packages/traitsui/editors/api.py", line 96 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 883 in exec_module
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "/home/runner/.local/lib/python3.10/site-packages/traitsui/editors/__init__.py", line 16 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 883 in exec_module
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 992 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "/home/runner/.local/lib/python3.10/site-packages/traitsui/api.py", line 257 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 883 in exec_module
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "/home/runner/.local/lib/python3.10/site-packages/mayavi/preferences/preference_manager.py", line 29 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 883 in exec_module
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "/home/runner/.local/lib/python3.10/site-packages/mayavi/preferences/api.py", line 4 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 883 in exec_module
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "/home/runner/.local/lib/python3.10/site-packages/mayavi/tools/engine_manager.py", line 12 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 883 in exec_module
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "/home/runner/.local/lib/python3.10/site-packages/mayavi/tools/camera.py", line 24 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 883 in exec_module
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "/home/runner/.local/lib/python3.10/site-packages/mayavi/mlab.py", line 16 in <module>
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 883 in exec_module
  File "<frozen importlib._bootstrap>", line 688 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 1006 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 1027 in _find_and_load
  File "<frozen importlib._bootstrap>", line 241 in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1078 in _handle_fromlist
  File "/home/runner/work/pyvirtualdisplay-bug/pyvirtualdisplay-bug/tests/test_animate.py", line 8 in <module>
  File "/home/runner/.local/lib/python3.10/site-packages/_pytest/assertion/rewrite.py", line 168 in exec_module
  ...
Extension modules: numpy.core._multiarray_umath, numpy.core._multiarray_tests, numpy.linalg.lapack_lite, numpy.linalg._umath_linalg, numpy.fft._pocketfft_internal, numpy.random._common, numpy.random.bit_generator, numpy.random._bounded_integers, numpy.random._mt19937, numpy.random.mtrand, numpy.random._philox, numpy.random._pcg64, numpy.random._sfc64, numpy.random._generator, traits.ctraits, PyQt5.QtCore, PyQt5.QtGui, PyQt5.QtWidgets, PyQt5.QtPrintSupport, PyQt5.QtNetwork, PyQt5.QtQml, PyQt5.QtBluetooth, PyQt5.QtDBus, PyQt5.QtDesigner, PyQt5.QtHelp, PyQt5.QtNfc, PyQt5.QtOpenGL, PyQt5.QtPositioning, PyQt5.QtLocation, PyQt5.QtQuick, PyQt5.QtQuick3D, PyQt5.QtQuickWidgets, PyQt5.QtRemoteObjects, PyQt5.QtSensors, PyQt5.QtSerialPort, PyQt5.QtSql, PyQt5.QtSvg, PyQt5.QtTest, PyQt5.QtTextToSpeech, PyQt5.QtWebChannel, PyQt5.QtWebSockets, PyQt5.QtX11Extras, PyQt5.QtXml, PyQt5.QtXmlPatterns (total: 44)
/home/runner/work/_temp/0585cdff-a041-431e-b28d-a33e621d22b2.sh: line 3:  6351 Aborted                 (core dumped) python3 -m pytest

Bug reproduction

I created a small repo to reproduce the issue. I created a bunch of branches where I cut some python dependencies off until I could isolate pyqt5 as the one breaking the working state: the changes are documented in the README. I know using external repos is not ideal, but since in that case it is a Github Action problem (it works on my local Ubuntu 22.04), maybe it makes sense?

are you looking for someone to debug this for you or ? what's the question — anthony sottile, Jul 05 '22 at 19:48
I am looking for someone who would be willing to at least indicate a direction, or a possible source of problem, or a similar issue they encountered that may be related. If this person is not you, that's fine with me. I have googled the error message, looked issues on related dependencies github pages, and I did not find anything relevant. The next logical step is to ask the community for feedback. — WaterFox, Jul 05 '22 at 19:53
you'll probably have to roll up your sleeves and use gdb to poke around and figure out what's going wrong — anthony sottile, Jul 05 '22 at 19:58
@gold_cy thank you for your question. I'm running tests for a [small library I develop](https://github.com/Becheler/quetzal-CRUMBS). Historically most of my installation problems came from scientific visualizations that have to run on headless servers (that's where PyVirtualDisplay, X11, pillow, mayavi etc come in). Deactivating all tests related to visualization allowed workflows to run correctly on MacOS/Ubuntu 20.04/Ubuntu 22.04. I got the headless settings work on remote MacOS and Ubuntu 20.04 github workflows and on my local Ubuntu 22.04. But I can't fix this remote Ubuntu 22.04 bug :/ — WaterFox, Jul 06 '22 at 01:27
unfortunately I don't think we have enough to go on to debug, you will have to take @AnthonySottile advice — gold_cy, Jul 06 '22 at 12:19
Tanks for trying! Yeah it's a weird bug and I struggle getting more info, even if I've been trying to narrow things done. The fact I can not reproduce the bug on local does not simplify the things either lol. I will give it a last try this morning, and if I can't find a solution I will remove Ubuntu 22 from the Github wokflows and see in few months/years if it works. — WaterFox, Jul 06 '22 at 13:10

WaterFox · Accepted Answer · 2022-07-06T21:39:19.777

So the problem is that the system dependencies of PyQt5 (the python wrapper for Qt) were not installed on Ubuntu 22.05.

I initially tried to install the required system dependency qt5-default (that is, what worked for Ubuntu 20.04), but as stated here and here, the qt5-default package was removed in Ubuntu 21.04 and at the time I assumed (misinformed decision) that it was because the package was now shipped with the distribution. Well, it's not ^^

The way around is to install the qt5-default dependencies, with:

sudo apt-get install qtbase5-dev qtchooser qt5-qmake qtbase5-dev-tools

That was enough to make the project run on Ubuntu 22.04 LTS, and it was obviously my mistake, but I wonder what in the error message could have put me on the right track (missing system dependency) a bit sooner.

Pytest core dump on Unbutu 22.04 LTS when using pyqt5 with mayavi and pyvirtualdisplay

Want I want to do:

Problem

Bug reproduction

1 Answers1