2

I have several python scripts which run using crontab every minute. After installing python-binance using pip3, only sometimes (seems random to me) all my scripts that use cryptography library get stuck at import.

As you can see from the log below, what was supposed to run every minute and last less than a second, at some point took more than ten minutes.

2019-03-30 08:51:07 INFO     (done)
2019-03-30 08:52:07 INFO     (done)
2019-03-30 08:53:07 INFO     (done)
2019-03-30 09:04:45 INFO     (done)
2019-03-30 09:05:45 INFO     (done)
2019-03-30 09:06:45 INFO     (done)

Trying to debug the problem, i run the script in a loop using python -vv.

During the slow iteration, the python interpreter was stuck here for several minutes

# trying /home/user/.local/lib/python3.6/site-packages/cryptography/hazmat/primitives/kdf/scrypt.cpython-36m-x86_64-linux-gnu.so
# trying /home/user/.local/lib/python3.6/site-packages/cryptography/hazmat/primitives/kdf/scrypt.abi3.so
# trying /home/user/.local/lib/python3.6/site-packages/cryptography/hazmat/primitives/kdf/scrypt.so
# trying /home/user/.local/lib/python3.6/site-packages/cryptography/hazmat/primitives/kdf/scrypt.py
# /home/user/.local/lib/python3.6/site-packages/cryptography/hazmat/primitives/kdf/__pycache__/scrypt.cpython-36.pyc matches /home/user/.local/lib/python3.6/site-packages/cryptography/hazmat/primitives/kdf/scrypt.py
# code object from '/home/user/.local/lib/python3.6/site-packages/cryptography/hazmat/primitives/kdf/__pycache__/scrypt.cpython-36.pyc'
import 'cryptography.hazmat.primitives.kdf.scrypt' # <_frozen_importlib_external.SourceFileLoader object at 0x7f5aff05d6a0>

Additional info:

  • OS: Ubuntu 18.04
  • Python version: python3 3.6.7-1~18.04

  • Python-binance version: python-binance==0.7.1

  • It's not a resource issue, when the interpreter was stuck the CPU was less then 20% used, plenty of RAM free and no disk bottleneck

  • The issue is occurring on different servers with the same configuration
  • I have tried to run the script using venv with the same result

UPDATE: After a suggestion on the python discord channel i have tried one more thing, which unfortunately didn't help

  • Increase the ulimit -n from 1024 to 4096

Any help is highly appreciated

TaTo
  • 46
  • 1
  • 8
  • The same here. Python 3.7.3, debian 9. Also only sometimes. Also virtualenv, but no such library installed. – Serge Apr 09 '19 at 10:36
  • pretty strange... i didn't have any problem before installing python-binance. Do you get stuck at the same point? did you try python3 -vv? – TaTo Apr 10 '19 at 00:01
  • yes exactly. there are a lot of modules installed but I installed them all at once so I am unable quickly find out which one causes the issue. – Serge Apr 10 '19 at 08:58
  • I just have opened the issue here – https://bugs.python.org/msg340722 – Serge Apr 23 '19 at 13:05
  • Hi Serge, thanks for letting me know! hopefully we will get some good advice how to troubleshoot further! – TaTo Apr 24 '19 at 11:22

1 Answers1

1

So after a long long troubleshooting, with some friend's support i figured out the problem was related to wheels.

i have installed the module in a venv using:

pip3 install binance --no-binary :all:

--no-binary :all: is making pip compile dependencies from source code instead of using any pre-compiled wheels potentially available.

Hope someone will find this solution helpful.

TaTo
  • 46
  • 1
  • 8