import numpy causes Python crash with NumPy version 1.16.0 and upwards

Question

Using a fresh installation of Python 3.7.3 on Windows Server 2012

import numpy causes a Python crash when NumPy versions 1.16.0 or above is installed. Downgrading back to 1.15.4 solves the problem, but I am failing to see the cause of this.

In addition, when I try to run the f2py.exe from the scripts directory directly, I get almost exactly the same crash error.

I tried downgrading one release at a time starting from 1.16.3, and the problem resolved at 1.15.4.

NumPy 1.16.0 release notes mention something like below, however I am only suspecting this could be relevant because the error mentions openBLAS as well

"The wheels on PyPI are linked with OpenBLAS v0.3.4+, which should fix the known threading issues found in previous OpenBLAS versions." https://docs.scipy.org/doc/numpy-1.16.0/release.html

I installed the packages using pip and did not build them myself.

Crash info following the import numpy command from within Python:

Problem signature:
  Problem Event Name:   APPCRASH
  Application Name: python.exe
  Application Version:  3.7.3150.1013
  Application Timestamp:    5c9954fa
  Fault Module Name:    libopenblas.IPBC74C7KURV7CB2PKT5Z5FNR3SIBV4J.gfortran-win_amd64.
  Fault Module Version: 0.0.0.0
  Fault Module Timestamp:   5cbcde25
  Exception Code:   c000001d
  Exception Offset: 00000000010fecc1
  OS Version:   6.3.9600.2.0.0.272.7
  Locale ID:    2057
  Additional Information 1: 42c8
  Additional Information 2: 42c87030f31f34adc07b43111a42b7a7
  Additional Information 3: f88e
  Additional Information 4: f88eb41d35291c2f266e7cad28e0f750

Crash info when f2py.exe is directly executed from Windows:

Problem signature:
  Problem Event Name:   APPCRASH
  Application Name: python.exe
  Application Version:  3.7.3150.1013
  Application Timestamp:    5c9954fa
  Fault Module Name:    libopenblas.IPBC74C7KURV7CB2PKT5Z5FNR3SIBV4J.gfortran-win_amd64.
  Fault Module Version: 0.0.0.0
  Fault Module Timestamp:   5cbcde25
  Exception Code:   c000001d
  Exception Offset: 00000000010fecc1
  OS Version:   6.3.9600.2.0.0.272.7
  Locale ID:    2057
  Additional Information 1: b03f
  Additional Information 2: b03fdfbd3d3cda6de0e3b052cf01b30d
  Additional Information 3: eca9
  Additional Information 4: eca9f354c1f2a69a54ab98e48c42f4c2

I would appreciate the help on how to resolve the issue, so I can upgrade NumPy to 1.16.x

Thank you very much !

EDIT:

stepping with

import pdb 
pdb.set_trace()
import numpy

from the IDE (vscode), this gets stuck at about here:

> c:\users\administrator\.vscode\extensions\ms-python.python-2019.4.12954\pythonfiles\lib\python\ptvsd\_vendored\pydevd\_pydevd_bundle\pydevd_net_command.py(77)__init__()
-> as_bytes = msg
(Pdb)
> c:\users\administrator\.vscode\extensions\ms-python.python-2019.4.12954\pythonfiles\lib\python\ptvsd\_vendored\pydevd\_pydevd_bundle\pydevd_net_command.py(78)__init__()
-> self._as_bytes = as_bytes
(Pdb)
--Return--
> c:\users\administrator\.vscode\extensions\ms-python.python-2019.4.12954\pythonfiles\lib\python\ptvsd\_vendored\pydevd\_pydevd_bundle\pydevd_net_command.py(78)__init__()->None
-> self._as_bytes = as_bytes
(Pdb)
--Return--
> c:\users\administrator\.vscode\extensions\ms-python.python-2019.4.12954\pythonfiles\lib\python\ptvsd\_vendored\pydevd\_pydevd_bundle\pydevd_net_command_factory_json.py(226)make_io_message()-><_pydevd_bund...000A095FB5240>
-> return NetCommand(CMD_WRITE_TO_CONSOLE, 0, event, is_json=True)
(Pdb)
> c:\users\administrator\.vscode\extensions\ms-python.python-2019.4.12954\pythonfiles\lib\python\ptvsd\_vendored\pydevd\pydevd.py(1773)write()
-> py_db.writer.add_command(cmd)
(Pdb)
--Call--
> c:\users\administrator\.vscode\extensions\ms-python.python-2019.4.12954\pythonfiles\lib\python\ptvsd\_vendored\pydevd\_pydevd_bundle\pydevd_comm.py(350)add_command()
-> def add_command(self, cmd):
(Pdb)
> c:\users\administrator\.vscode\extensions\ms-python.python-2019.4.12954\pythonfiles\lib\python\ptvsd\_vendored\pydevd\_pydevd_bundle\pydevd_comm.py(352)add_command()
-> if not self.killReceived:  # we don't take new data after everybody die
(Pdb)
> c:\users\administrator\.vscode\extensions\ms-python.python-2019.4.12954\pythonfiles\lib\python\ptvsd\_vendored\pydevd\_pydevd_bundle\pydevd_comm.py(353)add_command()
-> self.cmdQueue.put(cmd)
(Pdb)
--Call--
> c:\python37\lib\queue.py(121)put()
-> def put(self, item, block=True, timeout=None):
(Pdb)
> c:\python37\lib\queue.py(132)put()
-> with self.not_full:
(Pdb)
--Call--
> c:\python37\lib\threading.py(240)__enter__()
-> def __enter__(self):
(Pdb)
> c:\python37\lib\threading.py(241)__enter__()
-> return self._lock.__enter__()
(Pdb)
--Return--

directly from the command prompt:

Microsoft Windows [Version 6.3.9600]
(c) 2013 Microsoft Corporation. All rights reserved.

C:\Users\Administrator>python
Python 3.7.3 (v3.7.3:ef4ec6ed12, Mar 25 2019, 22:22:05) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import pdb
>>> pdb.set_trace()
--Return--
> <stdin>(1)<module>()->None
(Pdb) next
>>> import numpy
--Call--
> <stdin>(1)<module>()->None
(Pdb) step
> <stdin>(1)<module>()->None
(Pdb)
--Call--
> <frozen importlib._bootstrap>(978)_find_and_load()
(Pdb)
> <frozen importlib._bootstrap>(980)_find_and_load()
(Pdb)
--Call--
> <frozen importlib._bootstrap>(143)__init__()
(Pdb)
> <frozen importlib._bootstrap>(144)__init__()
(Pdb)
> <frozen importlib._bootstrap>(145)__init__()
(Pdb)
--Return--
> <frozen importlib._bootstrap>(145)__init__()->None
(Pdb)
--Call--
> <frozen importlib._bootstrap>(147)__enter__()
(Pdb)
> <frozen importlib._bootstrap>(148)__enter__()
(Pdb)
--Call--
> <frozen importlib._bootstrap>(157)_get_module_lock()
(Pdb)
> <frozen importlib._bootstrap>(163)_get_module_lock()
(Pdb)
> <frozen importlib._bootstrap>(164)_get_module_lock()
(Pdb)
> <frozen importlib._bootstrap>(165)_get_module_lock()
(Pdb)
> <frozen importlib._bootstrap>(166)_get_module_lock()
(Pdb)
KeyError: 'numpy'
> <frozen importlib._bootstrap>(166)_get_module_lock()
(Pdb)
> <frozen importlib._bootstrap>(167)_get_module_lock()
(Pdb)
> <frozen importlib._bootstrap>(168)_get_module_lock()
(Pdb)
> <frozen importlib._bootstrap>(170)_get_module_lock()
(Pdb)
> <frozen importlib._bootstrap>(171)_get_module_lock()
(Pdb)
> <frozen importlib._bootstrap>(174)_get_module_lock()
(Pdb)
--Call--
> <frozen importlib._bootstrap>(58)__init__()
(Pdb)
> <frozen importlib._bootstrap>(59)__init__()
(Pdb)
> <frozen importlib._bootstrap>(60)__init__()
(Pdb)
> <frozen importlib._bootstrap>(61)__init__()
(Pdb)
> <frozen importlib._bootstrap>(62)__init__()
(Pdb)
> <frozen importlib._bootstrap>(63)__init__()
(Pdb)
> <frozen importlib._bootstrap>(64)__init__()
(Pdb)
--Return--
> <frozen importlib._bootstrap>(64)__init__()->None
(Pdb)
> <frozen importlib._bootstrap>(176)_get_module_lock()
(Pdb)
> <frozen importlib._bootstrap>(187)_get_module_lock()
(Pdb)
> <frozen importlib._bootstrap>(189)_get_module_lock()
(Pdb)
> <frozen importlib._bootstrap>(191)_get_module_lock()
(Pdb)
--Return--
> <frozen importlib._bootstrap>(191)_get_module_lock()->_ModuleLock('... at 4331702088
(Pdb)
> <frozen importlib._bootstrap>(149)__enter__()
(Pdb)
--Call--
> <frozen importlib._bootstrap>(78)acquire()
(Pdb)
> <frozen importlib._bootstrap>(84)acquire()
(Pdb)
> <frozen importlib._bootstrap>(85)acquire()
(Pdb)
> <frozen importlib._bootstrap>(86)acquire()
(Pdb)
> <frozen importlib._bootstrap>(87)acquire()
(Pdb)
> <frozen importlib._bootstrap>(88)acquire()
(Pdb)
> <frozen importlib._bootstrap>(89)acquire()
(Pdb)
> <frozen importlib._bootstrap>(90)acquire()
(Pdb)
> <frozen importlib._bootstrap>(91)acquire()
(Pdb)
> <frozen importlib._bootstrap>(92)acquire()
(Pdb)
> <frozen importlib._bootstrap>(101)acquire()
(Pdb)
KeyError: 23192
> <frozen importlib._bootstrap>(101)acquire()
(Pdb)
--Return--
> <frozen importlib._bootstrap>(101)acquire()->None
(Pdb)
KeyError: 23192
> <frozen importlib._bootstrap>(101)acquire()->None
(Pdb)
--Return--
> <frozen importlib._bootstrap>(149)__enter__()->None
(Pdb)
KeyError: 23192
> <frozen importlib._bootstrap>(101)acquire()->None
(Pdb)
--Return--
> <frozen importlib._bootstrap>(980)_find_and_load()->None
(Pdb)
KeyError: 23192
> <frozen importlib._bootstrap>(101)acquire()->None
(Pdb)
--Return--
> <stdin>(1)<module>()->None
(Pdb)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<frozen importlib._bootstrap>", line 980, in _find_and_load
  File "<frozen importlib._bootstrap>", line 149, in __enter__
  File "<frozen importlib._bootstrap>", line 101, in acquire
KeyError: 23192
>>>
--Call--
> <stdin>(1)<module>()->None
(Pdb)
> <stdin>(1)<module>()->None
(Pdb)
--Return--
> <stdin>(1)<module>()->None
(Pdb)
>>>
--Call--
> <stdin>(1)<module>()->None
(Pdb)
> <stdin>(1)<module>()->None
(Pdb)
--Return--
> <stdin>(1)<module>()->None
(Pdb)
>>>

Can you find the line of NumPy code where it crashes? It might be in an import statement, in which case you'd want to try running that import statement by itself. Does your installation of NumPy 1.15 use OpenBLAS or not? — John Zwinck, May 11 '19 at 07:42
@JohnZwinck I installed numpy using a simple `pip install numpy` and I don't have experience with openBLAS so I am afraid I don't know how to answer that part of your question. I could not find the exact line where it crashes. — Kocas, May 11 '19 at 08:09
Here's one simple way to figure it out: write a script that does `import pdb; pdb.set_trace(); import numpy`. Run the script. Then use the "next" and "step" commands to go through the "import numpy" statement until it crashes, and see where the crash occurs. — John Zwinck, May 11 '19 at 08:21
You may want to seek help from someone who can sit with you and use pdb or other debuggers to pinpoint the problem. — John Zwinck, May 12 '19 at 01:00

import numpy causes Python crash with NumPy version 1.16.0 and upwards

0 Answers0