3

I am executing a notebook on my laptop and I get the following error.

XXX lineno: 17, opcode: 120
---------------------------------------------------------------------------
SystemError                               Traceback (most recent call last)
Input In [3], in <cell line: 3>()
      1 gym = Gym(0, 0, 0, 0).from_dill(BACKUP)
      2 ticker = gym.api.returnTicker()
----> 3 gym.wallet = gym.get_wallet()
      4 plot_donut_gym_wallet(gym)
      5 plot_donut_gym_wallet_makers(gym)

File <ipython-input-3-1c4842a503bf>:17, in get_wallet(self)

SystemError: unknown opcode

As you can see, the error happens during a function call. The function itself is not the problem, if I define and run ithe function inside a cell it simply works. But importing the function from its own module leads to this error. I have looked around for hints, all the forum I have read are pointing to some problem with having multiple Python installations. However, I tried using new environments, both using venv and conda and I get the same error. The same code on other machines works, so it appear to be something related to my particular installation, but I can't figure out how to fix it. I tried reinstalling conda, making new envs, upgrading python. Iteted this with Python 3.8, 3.9 and 3.10, I always get the same error. Any help is very welcome.

Since the function is from a class previously serialized using dill, this dill-related issue may be relevant https://github.com/uqfoundation/dill/issues/438

alec_djinn
  • 10,104
  • 8
  • 46
  • 71
  • Can you show all the steps that you did in order to create the virtual env, together with output, so it becomes more clear what your situation is. Also, what OS do you use? – a_guest Jul 11 '22 at 08:19
  • I simply used `conda create -n myenv python=3.10` changing python version. I am on MacOS 11.6.7. Which output are you referring to? – alec_djinn Jul 11 '22 at 08:37
  • I'm referring to every single command that you typed into your terminal, starting with the creation of the virtual env and ending with the above mentioned SystemError. Otherwise it's very difficult to estimate what is actually happening or how to resolve this. – a_guest Jul 11 '22 at 08:41
  • I create the env, then I run jupyter notebook in that environment `conda activate myenv` followed by `jupyter notebook`. The I run the cells of the notebook and the posted error comes out when the `gym.get_wallet()` function is called. The problem is, the error is not related to the function itself, rather is a SystemError involving opcodes. – alec_djinn Jul 11 '22 at 09:32
  • 1
    If these are really all the steps that you performed then this is a hint at the problem. To my knowledge, a fresh conda environment doesn't come with `jupyter` pre-installed, so if you manage to run `jupyter notebook` without having to install it beforehand, then most likely it refers to some other installation of Python on your system which happens to be on the `PATH`. You could check `import sys; print(sys.executable)` inside the notebook to verify what it refers to. – a_guest Jul 11 '22 at 11:41

1 Answers1

3

Code objects (from functions and class methods) serialized with dill are not guaranteed to work across different Python versions, because the list of valid opcodes change from version to version. In cases where there is an opcode unknown to the interpreter deserializing ("unpickling") the code object, the functions and classes are successfully recreated, but fail to execute as in your example.

The code in your example may have been created in Python 2, or Python 3.11 beta... It's impossible to know just by looking. You should ask whoever created the pickle file.


Edit: If you don't have the source code of the functions and methods, you may be able to reconstruct their source using a decompiler like decompyle3 (or uncompyle6 for other Python versions) and re-generate these faulty code objects with another Python version.

An example in Python3.7:

>>> def f(x, *, verbose=False):
...     y = x*x
...     if verbose:
...         print(y)
...     return y
...
>>> import inspect
>>> f_sign = inspect.signature(f)
<Signature (x, *, verbose=False)>
>>> 
>>> import os
>>> import decompyle3
>>> with open(os.devnull, 'w') as null:
...     f_code = decompyle3.main.decompile(f.__code__, out=null)
... 
>>> f_code
<decompyle3.semantics.pysource.SourceWalker object at 0x7f8932f14990>
>>> 
>>> from textwrap import indent
>>> print("def {}{}:\n{}".format(f.__name__, f_sign, indent(f_code.text, "    ")))
def f(x, *, verbose=False):
    y = x * x
    if verbose:
        print('x**2 =', y)
    return y

leogama
  • 898
  • 9
  • 13
  • Indeed, the class was serialized using dill with Python 3.7, it doesn't work with 3.8 and 3.9. Do you have any suggestion on how to port it on future version of python? Right now I am stuck with Python 3.7 to run that object from its backup. – alec_djinn Aug 10 '22 at 09:29
  • @alec_djinn see my edition to the answer – leogama Aug 11 '22 at 21:36
  • Wow, this is mind blowing, I was looking for something like this. Thanks! – alec_djinn Aug 12 '22 at 08:02
  • I'm glad it helped. If you are able to recreate the problematic functions but have difficulties replacing them in the unpickled objects context, please call me. Maybe I can give you some more directions. – leogama Aug 12 '22 at 16:59