3

I have a python shelve file generated with python 3.4 and UTF-8 encoding. The file can be opened and accessed fine on the host. When I access the file inside a python:3.4 Docker container, I get

>>> shelve.open('data/countries.shelf')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.4/shelve.py", line 243, in open
    return DbfilenameShelf(filename, flag, protocol, writeback)
  File "/usr/local/lib/python3.4/shelve.py", line 227, in __init__
    Shelf.__init__(self, dbm.open(filename, flag), protocol, writeback)
  File "/usr/local/lib/python3.4/dbm/__init__.py", line 94, in open
    return mod.open(file, flag, mode)
_gdbm.error: Bad magic number

It does not make any difference whether the file is bind-mounted into the container or actually copied during build, the error is the same.

I also compared the md5sum of the file outside and inside the container and they are identical (as I was expecting). But if the files are identical, how can the magic number be different?

Michael
  • 31
  • 1
  • 2
    You probably need to use identical python versions, not just matching major versions. – jordanm Dec 12 '17 at 16:33
  • 1
    Also, consider serializing your data into a format such as json instead of using shelve. – jordanm Dec 12 '17 at 16:34
  • Opening the same file on the host using python 3.6 works flawlessly. So I doubt it's a version issue. Using python:3.6 docker however fails again. I can clearly link the issue to the fact that python is running inside a container. – Michael Dec 12 '17 at 20:10
  • 3.6 is a major version. I meant for example 3.6.2 vs 3.6.4. – jordanm Dec 12 '17 at 21:48
  • Sorry for being unspecific in my comment. My point was that even between major versions there is no incompatibility of shelve files on the host. I guess what I am saying is that the python version does not make a difference (on the host system) and the problem only occurs inside a docker container (independent of the python version) which led me to the conclusion that the problem is caused by dockerizing things. – Michael Dec 13 '17 at 08:55

1 Answers1

0

Following from a comment by @jordanm, this is likely due to the cache file being created using a different python interpreter version than the one reading it.

Make sure you are writing and reading using the same interpreter version.

In my case, since the writing code is also the reading code, simply deleting the cache file was enough (thus re-creating it with the correct version).

Gulzar
  • 23,452
  • 27
  • 113
  • 201