0

I'm running a bunch of models. Because the code takes a long time to run, I pickle a few objects so I can load them back in later without having to redo everything. This is the code that I use to pickle and unpickle my objects:

def export_model(m, fname='model.pkl'):
    f = open(fname, 'wb')
    pickle.dump(m, f)
    f.close()


def import_model(fname):
    file = open(fname, 'rb')
    m = pickle.load(file)
    return m

It seems pretty straightforward and it's worked without issues on my Macbook. The problem comes when I try to run the code on a separate Windows machine. I can re-import the objects without any issues on the Windows computer. But when I try to use those same .pkl files to my Mac, pickle.load gives the following error:

TypeError: __randomstate_ctor() takes from 0 to 1 positional arguments but 2 were given

I checked that both computers are using the same format version:

print(pickle.format_version)
>>> 4.0

I'm a bit puzzled since the only difference is that the Windows machine is running Python 3.11.3 while the Mac is running 3.8.8. The code running on the two machines is the same (it's the same repository) so there shouldn't be any other discrepancies. Is this a Python version issue or a cross-platform issue?

caterp1llar
  • 1
  • 1
  • Are you sure you are opening the file in binary mode on both platforms? If you don't then the LF (Unix) versus CRLF (or NL vs CRNL) line ending difference will cause problems. Beyond that while the pickle format is supposedly architecture neutral I would check if your two systems have different architectures; e.g., x86_64 versus arm64. The last time I used Python's pickle module was long ago when all my systems were x86_64. – Kurtis Rader Jul 22 '23 at 05:44
  • According to this: https://stackoverflow.com/questions/1849523/is-pickle-file-of-python-cross-platform pickle is cross-platform and backwards compatible. By implication, it may not be forwards compatible – DarkKnight Jul 22 '23 at 05:51

1 Answers1

0

It appears that @DarkKnight was correct, this had to do with the difference in Python versions. I solved the problem by creating an environment with Python 3.11.3 on my Mac. Loading the pickle files back in no longer caused the error. Thanks!

caterp1llar
  • 1
  • 1