Stack Overflow
Stack Overflow

Questions tagged [pickle]

Pickle is an object serialization module for Python. Use this tag together with the Python tag for questions related to storing or loading objects with Pickle.

Pickle provides a powerful set of tools for serializing and de-serializing Python objects.

4936 questions
2
votes
3 answers

Python: how to convert pickled txt files into gpickles for networkx?

After working with a number of different occurrences of the same graph G, I dumped them as txt files with pickle using this line: pickling=pickle.dump(G,open('pickled_G.txt','w')) #Example for one single graph Now, for purposes of further…
FaCoffee
  • 7,609
  • 28
  • 99
  • 174
2
votes
1 answer

How do I unpickle a series of objects from a file in Python?

I have pickled objects to a file in append mode but it only reads a single object. Here's the code. I don't know what I am doing wrong. with open('notes.pkl', 'ab') as fileObject: #append pickle.dump(obj, fileObject,…
Me95
  • 45
  • 1
  • 4
2
votes
1 answer

python pickle.load() pkl file throws EOFError

a friend generates a pickle file for me. As he works on python 3 while I work on python 2, so he generated the file with pickle.dump(some_file_to_be_pickled,open("path_to_that_file","wb+"),protocol=2) While I tried to load it with the_file =…
gladys0313
  • 2,569
  • 6
  • 27
  • 51
2
votes
1 answer

unpickle in reverse order

Is there any way to unpickle multiple objects in reverse in order to load pickled objects from the last one pickled? For example: import pickle a = 'string1' b = 'string2' with open('test', 'wb') as f: pickle(a, f) pickle(b, f) After…
itf
  • 559
  • 1
  • 3
  • 15
2
votes
1 answer

Are there any metrics, known issues, or directions for saving a large pickle object to the Windows 10 file system

I have a rather large list: 19 million items in memory that I am trying to save to disk (Windows 10 x64 with plenty of space). pickle.dump(list, open('list.p'.format(file), 'wb')) Background: The original data was read in from a csv (2 columns)…
Pouya Barrach-Yousefi
  • 1,207
  • 1
  • 13
  • 27
2
votes
1 answer

Unpickle a cloudpickle without cloudpickle installation

The default pickle module from the Python standard library does not allow for the serialization of functions with closures, lambdas, or functions in __main__ (see here). I need to pickle an object using some custom functions that will not be…
Sealander
  • 3,467
  • 4
  • 19
  • 19
2
votes
1 answer

Python Pickle not saving entire object

I'm trying to pickle out a list of objects where the objects contain a list. When I open the pickled file I can see any data in my objects except from the list. I'm putting code below so this makes more sense. Object that contains a list. class…
user1904898
  • 59
  • 4
  • 18
2
votes
1 answer

Why can't python module dill pickle the generator function?

Dill module is awesome, it can pickle the entire python interpretor global state. But Dill cannot pickle generators. I know it is not supported. But i would like to know the technical reason given that it can pickle python functions?
AravindR
  • 677
  • 4
  • 11
2
votes
0 answers

Unpickling huge pickle files

I have huge pickle file around 6 GB generated for RainForestClassifer training samples using joblib.dump(). Each and every execution has to load the pickle objects using joblib.load() for processing the input data. The loading time is very high and…
chandu
  • 403
  • 1
  • 6
  • 15
2
votes
2 answers

Pickle high scores and then print

I'm trying to pickle high scores and then print them. In the actual program, score is acquired from a simple trivia game. score = 10 name = input("Name: ") scores = [name, score] high_scores = open("high_scores.dat", "ab") pickle.dump(scores,…
Gal A.
  • 172
  • 1
  • 13
2
votes
2 answers

Store python classes as pickles in GAE?

I am porting a Python investing application to Google App Engine. Every market that you can trade in is a plugin: for example the stocks trading and FOREX trading are all plugins. The application stores the portfolio (which is a Portfolio class…
Theo
  • 21
  • 1
2
votes
2 answers

UnicodeDecodeError: 'ascii' codec can't decode byte 0xe6 in position 1206: ordinal not in range(128)

I have an array W containing float numbers. W.dtype = float32 type(W) = Then I pickle.dump() it into a mr.pkl file, pickle.dump(W, open("/home/mr.pkl", "wb")) but when I load it, pickle.load(open("/home/mr.pkl","rb")) an…
Yunshi
  • 43
  • 2
  • 5
2
votes
2 answers

Preset Variable with Pickle

import time from random import randint import pickle MaTC = 1 MaTC = pickle.dump(MaTC, open("MaTCFile.txt", "wb")) AnTC = 2 AnTC = pickle.dump(AnTC, open("AnTCFile.txt", "wb")) AuTC = 3 AuTC = pickle.dump(AuTC,…
Lil Jakers
  • 77
  • 1
  • 1
  • 4
2
votes
2 answers

Object Larger Than RAM

I'm writing Python code right now which gradually constructs a large dictionary (600 million elements) while constantly reading from the dict. I regularly write that object to a file using cPickle, and pick up where I left off when interrupted by…
Eric Gorlin
  • 319
  • 1
  • 8
2
votes
0 answers

pickle memory leak in python

I have a large list of lists of list of integers, the lists at different levels are of different length, otherwise I would have stored using numpy. I now store the last level of list of integers using array('i', listofinteger) to save memory. The…
zzz
  • 21
  • 2
1 2 3
…
99
100