How to save a list of ojects that contains a list as member to file by using pickle python?

Question

I have the following class:

class Document:
  _queryID  = ""
  _name = []
  _docId = ""

In the code I create a list called listDocuments. I fill the list with objects of type Document.

import pickle 
if __name__ == "__main__": 
   listDocuments = []
   for i in range(0,10):
      e = Document()
      e._queryID = str(i)
      e._docId   = str(i+1)
      e._name.append("h"+str(i))
      listDocuments.append(e)
   save_data(listDocuments, "Variations.dat")
   list = load_data("Variations.dat")

   for obj  in listEntities:
     print obj._queryID  + "==>"+ obj._docId
     for var in obj._name:
       print var +"  "
     print "----\n"

I used the following functions taken from Python: how to save a list with objects in a file:

#save list of variations
def save_data(data, path):
    with open(path, "wb") as f:
        pickle.dump(data, f)

#load list of variations (Correctd after comments thank you)
def load_data(var_file):
    try:
        with open(var_file) as f:
            listDocument = pickle.load(f)
    except:
        listDocument = []

    return listDocument

As output I can only print the _queryID. What am I missing? I cannot print the list _name nor the _docID

I am unable to reproduce to problem, it seems to work fine for me. Could you post the code you use to print the list `_name`? Also I'd avoid naming a variable `list` (in your second code block), and you are missing the `s` of `listDocument` in the `try` — d6bels, Mar 06 '15 at 08:48

jedwards · Accepted Answer · 2015-03-06T08:56:25.537

Okay, a few issues.

First:

def load_data(var_file):
    try:
        with open(var_file) as f:
            listDocument = pickle.load(f)
    except:
        listDocuments = []

    return listDocuments

You use both listDocument and listDocuments. (Note one has a trailing s). Also, you're using these the variable listDocuments in the outer program which is hiding your errors. Let's rename the variable inside the function to data like the save_data function does. We'll also rename var_file to path for the same consistency reason.

Consider:

def load_data(path):
    data = []
    try:
        with open(path) as f:
            data = pickle.load(f)
    except: pass

    return data

Next, you're using list which is a reserved built-in -- lets change that to lst. So now you have:

class Document:
    _queryID  = ""
    _name = []
    _docId = ""

def save_data(data, path):
    with open(path, "wb") as f:
        pickle.dump(data, f)

#load list of variations
def load_data(path):
    data = []
    try:
        with open(path) as f:
            data = pickle.load(f)
    except: pass

    return data

import pickle
if __name__ == "__main__":
    listDocuments = []
    for i in range(0,10):
        e = Document()
        e._queryID = str(i)
        e._docId   = str(i+1)
        e._name.append("h"+str(i))
        listDocuments.append(e)
    save_data(listDocuments, "Variations.dat")
    lst = load_data("Variations.dat")

    for elem in lst:
        print elem._queryID
        print elem._docId
        print elem._name
        print "---"

Which works, and prints:

0
1
['h0', 'h1', 'h2', 'h3', 'h4', 'h5', 'h6', 'h7', 'h8', 'h9']
---
1
2
['h0', 'h1', 'h2', 'h3', 'h4', 'h5', 'h6', 'h7', 'h8', 'h9']
---
2
3
['h0', 'h1', 'h2', 'h3', 'h4', 'h5', 'h6', 'h7', 'h8', 'h9']
---
3
4
['h0', 'h1', 'h2', 'h3', 'h4', 'h5', 'h6', 'h7', 'h8', 'h9']
---
4
5
['h0', 'h1', 'h2', 'h3', 'h4', 'h5', 'h6', 'h7', 'h8', 'h9']
---
5
6
['h0', 'h1', 'h2', 'h3', 'h4', 'h5', 'h6', 'h7', 'h8', 'h9']
---
6
7
['h0', 'h1', 'h2', 'h3', 'h4', 'h5', 'h6', 'h7', 'h8', 'h9']
---
7
8
['h0', 'h1', 'h2', 'h3', 'h4', 'h5', 'h6', 'h7', 'h8', 'h9']
---
8
9
['h0', 'h1', 'h2', 'h3', 'h4', 'h5', 'h6', 'h7', 'h8', 'h9']
---
9
10
['h0', 'h1', 'h2', 'h3', 'h4', 'h5', 'h6', 'h7', 'h8', 'h9']
---

EDIT

Per your edit, you have no listEntities variable -- I think you mean (the recently renamed) lst:

for obj in lst:
    print obj._queryID  + "==>"+ obj._docId
    for var in obj._name:
        print var +"  "
    print "----\n"

Lastly, if you're wondering why the _name variable is the same for all instances of the Document class, it's because you're declaring _name as a class attribute (so its the same for all instances), not an instance attribute.

You could replace the Document class definition with:

class Document:
    def __init__(self):
        self._name = []

And you'll probably be happier with the results:

0==>1
h0  
----

1==>2
h1  
----

2==>3
h2  
----

3==>4
h3  
----

4==>5
h4  
----

5==>6
h5  
----

6==>7
h6  
----

7==>8
h7  
----

8==>9
h8  
----

9==>10
h9  
----

I was wondering what am i missing. I didn't know that I should declare it a class attribute. I actually treated class as a structure just like in C++. Hope I am not wrong on that part too. Well I consider this answer as a **double** correction lololol — Hani Goc, Mar 06 '15 at 09:12
No worries, the way you wrote your `Document` definition is similar to the `static` concept in C++. Those attributes will be shared by every instance of the class. If you want independent instance attributes, you might put them inside the `__init__` method, with `self.` preceding them (like I wrote `self._name`). Note that in Python, unlike C++, you don't *need* to do this. In the case of `_name` you need to initialize it because you're `append`ing to it later, but in the case of `_queryID` and `_docId` you do not, which is why I omitted the initialization. (Exceptions/weirdness exists) — jedwards, Mar 06 '15 at 09:38

How to save a list of ojects that contains a list as member to file by using pickle python?

1 Answers1