2

[OS info: 5.14.14-arch1-1 x86_64 GNU/Linux]

After creating a Python v3.7.0 virtual-env with Pyenv, with the --no-site-packages option, I install packages with pip, using:

$ python -m -pip install <package-name>

An example is either Pandas v1.1.5 or Torch v1.7.0+cpu installed in the virtual-env. Then, starting a Python 3.7.0 shell in console, simply by issuing python in terminal in that environment, I try to import those packages. The result is:

$ python
Python 3.7.0 (default, Oct 20 2021, 09:16:31)
[GCC 11.1.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
+>>> import faulthandler; faulthandler.enable()
+>>> import torch
Fatal Python error: Segmentation fault
Current thread 0x00007fac608f22c0 (most recent call first):
  File "/home/USER/.pyenv/versions/3.7.0/lib/python3.7/ctypes/__init__.py", line 273 in _reset_cache
  File "/home/USER/.pyenv/versions/3.7.0/lib/python3.7/ctypes/__init__.py", line 543 in <module>
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 728 in exec_module
  File "<frozen importlib._bootstrap>", line 677 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 967 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 983 in _find_and_load
  File "/home/USER/.pyenv/versions/3.7.0/lib/python3.7/site-packages/torch/__init__.py", line 16 in <module>
  File "<frozen importlib._bootstrap>", line 219 in _call_with_frames_removed
  File "<frozen importlib._bootstrap_external>", line 728 in exec_module
  File "<frozen importlib._bootstrap>", line 677 in _load_unlocked
  File "<frozen importlib._bootstrap>", line 967 in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 983 in _find_and_load
  File "<stdin>", line 1 in <module>
Segmentation fault
$

The interesting thing is that if in that same virtual-env, I put together an iPython kernel, like so:

$ ipython kernel install --user --name <my_project> --display-name "<Python3.7.0 (my_project)>"

and import the same package(s), say from a Jupyter notebook running that kernel, there is absolutely no issue. This tells me that there is no version issue (although dependency issues could occur still, I suppose. The segmentation fault only occurs in the Python shell in console. Also when I try importing from my pyenv global environment (3.10.0), there seems to be no issue whatsoever.

Any help to understand why and how to resolve the issue is welcome.

Cbhihe
  • 511
  • 9
  • 24

1 Answers1

0

This is a tentative answer as I lack of a definite proof of solution. Anyone more comfortable than I am with debugging is welcome to add to it or to write their own answer.

(i) I originally thought something along the line of a corrupted ~/.inputrc might be responsible for the "hoopla" along the lines of this answer on SO. However nothing had changed in that ~/.inputrc file in past months. So the issue was more likely to be a problematic update sequence.

(ii) Looking at lines 273 and 543 in /home/USER/.pyenv/versions/3.7.0/lib/python3.7/ctypes/__init__.py the _reset_cache() function seems to trigger the segfault, just as module ctypes is imported for instance in ../torch/__init__.py.

(iii) updating pyenv to the latest version build (v2.1.0-1) and, a day later, gdbm aka GNU dbm (v1.22-1), the database function library on which Python depends, seems to have solved the issue.
(Co-)incidentally both packages are maintained by the same person. ;-)

Still unsure about exactly what was at play here. Consulting the GDBM documentation did not help. Sorry about the incomplete answer.

Cbhihe
  • 511
  • 9
  • 24