Unexpected error when Intel python3.7 shell is launched : impossible to do any command - abort error

Question

I get the following error when I do a simple command a=1 (or any other command) once an Intel python3 shell is launched on a Debian 10:

Python 3.7.11 (default, Sep  9 2021, 02:06:38)
[GCC 9.3.0] :: Intel Corporation on linux
Type "help", "copyright", "credits" or "license" for more information.
Intel(R) Distribution for Python is brought to you by Intel Corporation.
Please check out: https://software.intel.com/en-us/python-distribution
(.startup.py)
(imported numpy as np)
(imported matplotlib.pyplot as plt)
(imported scipy.interpolate CubicSpline)
(imported scipy.interpolate interp1d)
(imported os)
(imported sys)
(imported glob)
(imported re)
>>> a=1
free(): invalid pointer
zsh: abort      /opt/intel/oneapi/intelpython/python3.7/bin/python3.7

What is happening?

Permissions seem to be OK:

-rwxrwxr-x 1 19712 20M Sep  9 04:16 /opt/intel/oneapi/intelpython/python3.7/bin/python3.7

I don't understand.

Edit 1

A gdb on python3 allows to get more informations :

gdb python3
GNU gdb (Debian 8.2.1-2+b3) 8.2.1
Copyright (C) 2018 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Type "show copying" and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
    <http://www.gnu.org/software/gdb/documentation/>.

For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from python3...done.
(gdb) file python3
Load new symbol table from "python3"? (y or n) y
Reading symbols from python3...done.
(gdb) r
Starting program: /opt/intel/oneapi/intelpython/python3.7/bin/python3
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Python 3.7.11 (default, Sep  9 2021, 02:06:38)
[GCC 9.3.0] :: Intel Corporation on linux
Type "help", "copyright", "credits" or "license" for more information.
Intel(R) Distribution for Python is brought to you by Intel Corporation.
Please check out: https://software.intel.com/en-us/python-distribution
(.startup.py)
(imported numpy as np)
(imported matplotlib.pyplot as plt)
(imported scipy.interpolate CubicSpline)
(imported scipy.interpolate interp1d)
(imported os)
(imported sys)
(imported glob)
(imported re)
>>> a=1
free(): invalid pointer

Program received signal SIGABRT, Aborted.
__GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
50  ../sysdeps/unix/sysv/linux/raise.c: No such file or directory.
(gdb) bt
#0  __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:50
#1  0x00007ffff7c56535 in __GI_abort () at abort.c:79
#2  0x00007ffff7cad508 in __libc_message (action=action@entry=do_abort, fmt=fmt@entry=0x7ffff7db828d "%s\n")
    at ../sysdeps/posix/libc_fatal.c:181
#3  0x00007ffff7cb3c1a in malloc_printerr (str=str@entry=0x7ffff7db643b "free(): invalid pointer")
    at malloc.c:5341
#4  0x00007ffff7cb542c in _int_free (av=<optimized out>, p=<optimized out>, have_lock=<optimized out>)
    at malloc.c:4165
#5  0x00005555555f7abd in PyOS_Readline (sys_stdin=<optimized out>,
    sys_stdout=0x7ffff7df0760 <_IO_2_1_stdout_>, prompt=0x7ffff7656f20 ">>> ")
    at /home/sat_bot/base/conda-bld/python_1631149597897/work/Parser/myreadline.c:377
#6  0x0000555555704c02 in tok_nextc (tok=0x55555633ca00)
    at /home/sat_bot/base/conda-bld/python_1631149597897/work/Parser/tokenizer.c:939
#7  0x000055555572c017 in tok_get (p_end=<optimized out>, p_start=<optimized out>, tok=<optimized out>)
    at /home/sat_bot/base/conda-bld/python_1631149597897/work/Parser/tokenizer.c:1373
#8  PyTokenizer_Get (p_end=<optimized out>, p_start=<optimized out>, tok=<optimized out>)
    at /home/sat_bot/base/conda-bld/python_1631149597897/work/Parser/tokenizer.c:1835
#9  parsetok (tok=<optimized out>, g=<optimized out>, start=<optimized out>, err_ret=<optimized out>,
    flags=<optimized out>) at /home/sat_bot/base/conda-bld/python_1631149597897/work/Parser/parsetok.c:208
#10 0x00005555557cd310 in PyParser_ASTFromFileObject (fp=<optimized out>, filename=0x7ffff7778530,
    enc=<optimized out>, start=<optimized out>, ps1=<optimized out>, ps2=<optimized out>,
    flags=0x7fffffffd860, errcode=0x7fffffffd784, arena=0x7ffff783e390)
--Type <RET> for more, q to quit, c to continue without paging--
    at /home/sat_bot/base/conda-bld/python_1631149597897/work/Python/pythonrun.c:1222
#11 0x0000555555640ee6 in PyRun_InteractiveOneObjectEx (fp=0x7ffff7defa00 <_IO_2_1_stdin_>,
    filename=0x7ffff7778530, flags=0x7fffffffd860)
    at /home/sat_bot/base/conda-bld/python_1631149597897/work/Python/pythonrun.c:236
#12 0x00005555556411cc in PyRun_InteractiveLoopFlags (fp=0x7ffff7defa00 <_IO_2_1_stdin_>,
    filename_str=<optimized out>, flags=0x7fffffffd860)
    at /home/sat_bot/base/conda-bld/python_1631149597897/work/Python/pythonrun.c:120
#13 0x00005555557cd85e in PyRun_AnyFileExFlags (fp=0x7ffff7defa00 <_IO_2_1_stdin_>,
    filename=0x5555557ce4b8 "<stdin>", closeit=0, flags=0x7fffffffd860)
    at /home/sat_bot/base/conda-bld/python_1631149597897/work/Python/pythonrun.c:78
#14 0x00005555557cdbdc in pymain_run_file (p_cf=0x7fffffffd860, filename=<optimized out>,
    fp=0x7ffff7defa00 <_IO_2_1_stdin_>)
    at /home/sat_bot/base/conda-bld/python_1631149597897/work/Modules/main.c:462
#15 pymain_run_filename (cf=0x7fffffffd860, pymain=0x7fffffffd970)
    at /home/sat_bot/base/conda-bld/python_1631149597897/work/Modules/main.c:1655
#16 pymain_run_python (pymain=0x7fffffffd970)
    at /home/sat_bot/base/conda-bld/python_1631149597897/work/Modules/main.c:2916
#17 pymain_main (pymain=0x7fffffffd970)
    at /home/sat_bot/base/conda-bld/python_1631149597897/work/Modules/main.c:3456
#18 0x00005555557cdd2c in _Py_UnixMain (argc=<optimized out>, argv=<optimized out>)
    at /home/sat_bot/base/conda-bld/python_1631149597897/work/Modules/main.c:3491
#19 0x00007ffff7c5809b in __libc_start_main (main=0x555555641820 <main>, argc=1, argv=0x7fffffffdac8,
--Type <RET> for more, q to quit, c to continue without paging--
    init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7fffffffdab8)
    at ../csu/libc-start.c:308
#20 0x0000555555733be9 in _start ()
    at /home/sat_bot/base/conda-bld/python_1631149597897/work/Parser/parser.c:325
(gdb)

Are there any useful clues among all this information?

Edit 2

If I launch the shell as root, there isn't this issue:

# python3.7
Python 3.7.11 (default, Sep  9 2021, 02:06:38)
[GCC 9.3.0] :: Intel Corporation on linux
Type "help", "copyright", "credits" or "license" for more information.
Intel(R) Distribution for Python is brought to you by Intel Corporation.
Please check out: https://software.intel.com/en-us/python-distribution
(.startup.py)
(imported numpy as np)
(imported matplotlib.pyplot as plt)
(imported scipy.interpolate CubicSpline)
(imported scipy.interpolate interp1d)
(imported os)
(imported sys)
Traceback (most recent call last):
  File "/root/.startup.py", line 16, in <module>
    import readline
ImportError: /opt/intel/oneapi/intelpython/python3.7/lib/python3.7/site-packages/readline.cpython-37m-x86_64-linux-gnu.so: undefined symbol: tputs
>>> a=1
>>> a=np.arange(10)
>>> a
array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
>>>

Edit 3

I kept researching on this issue.

If I do, like suggested:

$ sudo python3.7
Python 3.7.3 (default, Jan 22 2021, 20:04:44)
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> a=range(10)
>>> a
range(0, 10)
>>>

Then, I have no errors but all libraries from ~/.startup.py:

print("(.startup.py)")
import numpy as np
print("(imported numpy as np)")
import matplotlib.pyplot as plt
print("(imported matplotlib.pyplot as plt)")
from scipy.interpolate import CubicSpline
print("(imported scipy.interpolate CubicSpline)")
from scipy.interpolate import interp1d
print("(imported scipy.interpolate interp1d)")
import atexit
print("(imported os)")
import os
print("(imported sys)")
import sys
import readline
import rlcompleter
print("(imported glob)")
import glob
print("(imported re)")
import re

are not available.

I think the conflict comes from the fact that with $ sudo pthon3.7, I call the binary /usr/bin/python3.7 and not /opt/intel/oneapi/intelpython/python3.7/bin/python3.7.

How to make distinguish the two versions? or maybe how could I avoid conflicts between both?

I mention also that with /opt/intel/oneapi/intelpython/python3.7/bin/python3.7, the python shell displays:

Python 3.7.11 (default, Sep  9 2021, 02:06:38)
[GCC 9.3.0] :: Intel Corporation on linux

That would mean that GCC-9.3.0 is installed on my system but this is not actually true: maybe this Intel python expects a gcc-9.3 which is not basically installed.

Is this a good track?

We'd need to inspect the binary, compare the libc and other libraries on your system to the ones it was compiled against, etc. — Charles Duffy, Oct 05 '21 at 21:09
That said, something you could do to collect more information yourself would be to run it under gdb and collect a stack trace. — Charles Duffy, Oct 05 '21 at 21:10
Try removing (or renaming) the `PYSTARTUP` environment variable. — John Gordon, Oct 05 '21 at 21:11
In general, using 3rd-party binaries is not a great idea. Your performance might be better with the Intel version, but for reliability and compatibility I'd stick to using a Python runtime compiled by your Linux distro, or at least compiled _on the same version of_ your Linux distro, against the same libraries it ships and was tested with. (An exception is using a 3rd-party package manager like Nix, since it ships a complete ecosystem of dependencies down to the libc, with rpaths compiled in to ensure that the libraries it shipped are actually used). — Charles Duffy, Oct 05 '21 at 21:12
I'd maybe ask over at their forum - https://community.intel.com/t5/Intel-Distribution-for-Python/bd-p/distribution-python — unddoch, Oct 05 '21 at 21:14
By the way, you might also make sure `LD_LIBRARY_PATH` isn't set. If Intel's Python distro does ship its own libc, having something linked against your system libc forcefully loaded is a quick route to surprises. — Charles Duffy, Oct 05 '21 at 21:15
@CharlesDuffy . Which is the appropriate `LD_LIBRARY_PATH` value to set ? — , Oct 05 '21 at 21:27
Ideally there should be none at all, unless Intel's documentation specifies one to use. — Charles Duffy, Oct 05 '21 at 22:32
And it working when you run as root doesn't necessarily mean there's an installation problem. Using `sudo` clears a lot of environment variables, _including_ `LD_LIBRARY_PATH` and `LD_PRELOAD`, for security reasons (and, also for security reasons, makes some other variables be ignored). So it could very easily be a user-environment problem triggered by a setting that isn't there for root's account. — Charles Duffy, Oct 05 '21 at 22:34
@youpilat13: I have removed the answer that was merged into the question. By all means, do post an answer. Click on the blue "Answer Your Question" button at the bottom of the page. — halfer, Oct 07 '21 at 21:35

score 1 · Accepted Answer · answered Oct 13 '21 at 05:18

When you source setvars for oneAPI(or intel python) the intel python executable gets added to the system path of current user.

The path variable would be an environment variable specific to your current user. Thus when you execute a command with sudo the environment variables are not considered this is why you are executing 2 different python executables with and without sudo.

You could verify this with the below commands

sudo which python
which python

If you need to use the same python executable as sudo and normal user, Run sudo -E python to launch the shell to have the same binary as that of your normal user. (-E preserves environment variables)

The gcc version in the python interpreter output just refers to the gcc version that was used to built the python binary. It has not got anything to do with the gcc you have installed.

The error in your query would be due to having user packages in your pip environment which might be incompatible with each other. Could you create a new environment with intel python, I would suggest using conda if you have anaconda installed and check if you still get the error. Let me know if you are still facing issues. PFB a link which guides on installation process.

https://software.intel.com/content/www/us/en/develop/articles/using-intel-distribution-for-python-with-anaconda.html

Unexpected error when Intel python3.7 shell is launched : impossible to do any command - abort error

Edit 1

Edit 2

Edit 3

1 Answers1