python shelve is not saving/loading

Question

When I save/load my workspace via functions in a subfile, shelve doesn't work (test1). However, if I do the same in one file, it works (test2). Why is that? How can I fix the problem for the first case?

In the main file:

# in saveWS.py   

# to save
def saveSlv(fileName):
    destination='./'+fileName+'_shelve.pkl'
    bk = shelve.open(destination,'n')
    for k in dir():
        try:
            bk[k] = globals()[k]
        except Exception:
            pass
    bk.close()
        
# to restore
def loadSlv(fileName):
    myshelve ='./'+fileName+'_shelve.pkl'
    bk_restore = shelve.open(myshelve)
    for k in bk_restore:
        globals()[k] = bk_restore[k]
    bk_restore.close()

In the main file:

import shelve
# User defined functions
from saveWS import saveSlv, loadSlv 

# It doesn't work
a=1,2,3
b='ypk'
fileName='test1'
# save the variables in work space by calling a function
saveSlv(fileName)
del a, b
# restore the work space by calling a function
loadSlv(fileName)
 
# It works
a=1,2,3
b='ypk'
fileName='test2'

# save the variables in work space
destination='./'+fileName+'_shelve.pkl'
bk = shelve.open(destination,'n')
for k in dir():
    try:
        bk[k] = globals()[k]
    except Exception:
        pass
bk.close()
del a, b

# restore the work space
myshelve ='./'+fileName+'_shelve.pkl'
bk_restore = shelve.open(myshelve)
for k in bk_restore:
    globals()[k] = bk_restore[k]
bk_restore.close()

Timus · Accepted Answer · 2022-03-15T11:55:37.767

Here's the globals()-related part of your problem (I guess):

Return the dictionary implementing the current module namespace. For code within functions, this is set when the function is defined and remains the same regardless of where the function is called.

So globals() in your functions is always the namespace of saveWS.py.

And here the dir()-related one:

Without arguments, return the list of names in the current local scope. With an argument, attempt to return a list of valid attributes for that object.

Therefore dir() refers to the local namespace within the function.

You probably could fix that by passing dir() and globals() as arguments:

def saveSlv(fileName, variables, namespace):
    destination='./'+fileName+'_shelve.pkl'
    bk = shelve.open(destination,'n')
    for k in variables:
        try:
            bk[k] = namespace[k]
        except Exception:
            pass
    bk.close()

def loadSlv(fileName, namespace):
    myshelve ='./'+fileName+'_shelve.pkl'
    bk_restore = shelve.open(myshelve)
    for k in bk_restore:
        namespace[k] = bk_restore[k]
    bk_restore.close()

Calling:

saveSlv(fileName, dir(), globals())
loadSlv(fileName, globals())

You could do without the variables argument by using

...
    for k in dir(sys.modules[namespace['__name__']]):
        ...

instead (after importing sys). In your usecase you could probably further replace namespace['__name__'] with '__main__'. If you want to get rid of all the additional arguments you could try if this works for you:

import shelve
import sys

def saveSlv(fileName):
    destination='./'+fileName+'_shelve.pkl'
    bk = shelve.open(destination,'n')
    namespace = sys.modules['__main__'].__dict__
    for k in dir(sys.modules['__main__']):
        try:
            bk[k] = namespace[k]
        except Exception:
            pass
    bk.close()


def loadSlv(fileName):
    namespace = sys.modules['__main__'].__dict__
    myshelve ='./'+fileName+'_shelve.pkl'
    bk_restore = shelve.open(myshelve)
    for k in bk_restore:
        namespace[k] = bk_restore[k]
    bk_restore.close()

Check here if that's appropriate.

Thanks for the detailed explanations & insightful examples! – yufiP Mar 16 '22 at 03:29 — yufiP, Mar 16 '22 at 03:29

python shelve is not saving/loading

1 Answers1