Stack Overflow
Stack Overflow

Questions tagged [shelve]

Shelve can refer to either a Python object persistence or a pending change in a Visual Studio Workspace

A shelf in Python is a persistent, dictionary-like object. Documentation: shelve

Shelve can also refer to a pending change in the workspace within Visual Studio 2005

351 questions
8
votes
3 answers

Can Python's shelve.open be called in a nested fashion?

I am trying to write a memoization library that uses shelve to store the return values persistently. If I have memoized functions calling other memoized functions, I am wondering about how to correctly open the shelf file. import shelve import…
saul.shanabrook
  • 3,068
  • 3
  • 31
  • 49
8
votes
3 answers

Shelve module in python not working: "db type cannot be determined"

I am trying to make a simple password-storing program in Python, and it seems pretty simple so I am wondering if I am using shelve wrong. I have the main .py file: import shelve passwords = shelve.open('./passwords_dict.py') choice =…
yung galbanum
  • 329
  • 6
  • 11
8
votes
3 answers

Shelve: choice of database

shelve documentation says: The choice of which database package will be used (such as dbm, gdbm or bsddb) depends on which interface is available. What is that mean? How to determine which package choosen? How to strictly define which one must…
Gill Bates
  • 14,330
  • 23
  • 70
  • 138
7
votes
1 answer

How to populate shelf with existing dictionary

Lets say I have a large, 100s of megabytes, dictionary that I want to make into an on disk shelve. I'm using pypar to utilize MPI to generate cleaned bits of a master list. What's the best way to achieve this? Example: # much earlier masterDict =…
Martlark
  • 14,208
  • 13
  • 83
  • 99
7
votes
0 answers

Shelve out of overflow pages on macos sierra

While writing to a shelve object I encountered a problem, where I cannot write more records. I get HASH: Out of overflow pages. Increase page size followed by _dbm.error: cannot add item to database. Problem occurs on both macos Sierra and earlier…
fulaphex
  • 2,879
  • 3
  • 19
  • 26
7
votes
1 answer

Is shelve unsafe for read-only multiprocessing?

The documentation of the shelve module makes the following claim under Restrictions: The shelve module does not support concurrent read/write access to shelved objects. (Multiple simultaneous read accesses are safe.) As far as I can tell, this…
Henry Keiter
  • 16,863
  • 7
  • 51
  • 80
7
votes
3 answers

Persistent multiprocess shared cache in Python with stdlib or minimal dependencies

I just tried a Python shelve module as the persistent cache for data fetched from the external service. The complete example is here. I was wondering what would the best approach if I want to make this multiprocess safe? I am aware of redis,…
Mikko Ohtamaa
  • 82,057
  • 50
  • 264
  • 435
6
votes
2 answers

How to keep uncommitted changes in a local mercurial repository, while still pushing/pulling?

If i'm working on some files that I don't want to commit, I just save them. I then have other files I want to push to the server, however if someone else has made changes to the repository, and I pull them down, it asks me to merge or rebase.. but…
Jason
  • 363
  • 3
  • 14
6
votes
0 answers

Reduce Python shelve size

I'm using the shelve module with gdbm to store Python objects. I understand that shelve uses pickle to store the objects. Unfortunately, the sizes of my shelves are too large. I found this solution to bzip2 or gzip individual pickles. My question…
Botond
  • 2,640
  • 6
  • 28
  • 44
6
votes
1 answer

python export env variable in Linux shell

I tried to export path to system environment and what happens: Path is for BerkleyDB library (need for shelve Python library, without it I have an error: File "/kplusnfs/script/ns/processes/current/kondor_processes.py", line 214, in…
Tomasz Wójcik
  • 61
  • 1
  • 1
  • 6
6
votes
2 answers

How do you erase the 3 shelve files in python 3?

I wrote a few unittests with shelve at http://code.google.com/p/filecache/ and python 2 saves exactly the filename I specifiy in shelve.open() but in python 3 I get 3 different files "bak", "dat" and "dir". So before the tests start I want to erase…
ubershmekel
  • 11,864
  • 10
  • 72
  • 89
6
votes
2 answers

UnpicklingError: pickle data was truncated when trying to read a dictionary from a shelved file

I'm a teacher, and I'm trying to write a simple function that saves my students' emails in a dictionary for use in another program. I need the dictionary to be saved across multiple executions, so I'm trying to use shelve to save it; however, after…
J. Browning
  • 61
  • 1
  • 1
  • 3
6
votes
3 answers

Is there anything wrong with creating a Python Pickle powered website?

I have been toying with this idea for quite awhile now, but haven't seen any information on people doing it. I have a small website project where I need to load and modify 1 object. This object is pretty simple, and shouldn't be more than a few kb.…
Shane Reustle
  • 8,633
  • 8
  • 40
  • 51
6
votes
2 answers

Use "hg unshelve" like unstashing with Git

When invoking hg unshelve --keep and I get a conflict, I need to resolve the conflict and then invoke hg unshelve --continue --keep again. Why is the last step necessary? And why I can't invoke hg unshelve --continue --keep directly without…
Thomas S.
  • 5,804
  • 5
  • 37
  • 72
6
votes
2 answers

Shelve dictionary size is >100Gb for a 2Gb text file

I am creating a shelve file of sequences from a genomic FASTA file: # Import necessary libraries import shelve from Bio import SeqIO # Create dictionary of genomic sequences genome = {} with open("Mus_musculus.GRCm38.dna.primary_assembly.fa") as…
jma1991
  • 345
  • 1
  • 3
  • 15
1 2
3
…
23 24