How to use mock_open with pickle.load

Question

I want to implement unit test for a function :

def load_pickle(path):
    with open(path, 'r') as of:
        return pickle.load(of)

I adapted what I found on https://nickolaskraus.org/articles/how-to-mock-the-built-in-function-open/ to implement my unittest

import pickle
from mock import mock_open

def test_load_pickle(self):
        read_data = pickle.dumps({'a': 1, 'b': 2, 'c': 3})
        mockOpen = mock_open(read_data=read_data)
        with patch('__builtin__.open', mockOpen):
            # 'testPath' is supposed to be just a string that doesn't correspond to any real path
            obj = load_pickle('testPath')
        self.assertEqual({'a': 1, 'b': 2, 'c': 3}, obj)

While is works well when I change all pickle to json, it doesn't work with pickle and I have an error :KeyError: "(dp0\nS'a'\np1\nI1\nsS'c'\np2\nI3\nsS'b'\np3\nI2\ns."

The issues is that the pickle.load part doesn't work. I have searched online and didn't find any solution. Don't you have any idea of how to adapt my code to use mock_open with pickle.load ? Thanks !

PS : I'm using python27 (I know it's bad but my company didn't migrate to python3 yet)

EDIT : My full code

import unittest
import pickle

from mock import mock_open, patch

class Class(object):
    def load_pickle(self, path):
        with open(path, 'rb') as of:
            return pickle.load(of)


class TestClass(unittest.TestCase):
    def test_load_pickle(self):
        read_data = pickle.dumps({'a': 1, 'b': 2, 'c': 3})
        mockOpen = mock_open(read_data=read_data)
        with patch('__builtin__.open', mockOpen):
            obj = Class().load_pickle('testPath')
        self.assertEqual({'a': 1, 'b': 2, 'c': 3}, obj)

Full traceback :

Traceback (most recent call last):
  File "/home/lsoret/softwares/miniconda3/envs/py27/lib/python2.7/unittest/case.py", line 329, in run
    testMethod()
  File "/home/lsoret/code/qynapse/python/qia/tests/models/testtest.py", line 17, in test_load_pickle
    obj = Class().load_pickle('testPath')
  File "/home/lsoret/code/qynapse/python/qia/tests/models/testtest.py", line 9, in load_pickle
    return pickle.load(of)
  File "/home/lsoret/softwares/miniconda3/envs/py27/lib/python2.7/pickle.py", line 1384, in load
    return Unpickler(file).load()
  File "/home/lsoret/softwares/miniconda3/envs/py27/lib/python2.7/pickle.py", line 864, in load
    dispatch[key](self)
KeyError: "(dp0\nS'a'\np1\nI1\nsS'c'\np2\nI3\nsS'b'\np3\nI2\ns."

Hm, looks fine to me. I just tested it (putting a test class around, as you seem to use `unittest`), and the test passes fine. Can you please show the whole source (complete with test class)? — MrBean Bremen, Mar 19 '20 at 18:34
I ran the very same code (with a test class arround) and it doesn't work, I have a `KeyError: "(dp0\nS'a'\np1\nI1\nsS'c'\np2\nI3\nsS'b'\np3\nI2\ns."`. Maybe it's because I'm using Python2.7 ? — luc.soret, Mar 20 '20 at 07:51
I used Python 2.7, too. Can you please show your complete test code? — MrBean Bremen, Mar 20 '20 at 08:04
Hm, works fine for me... what system you are on? I tested under Windows 10, with Python 2.7.16. — MrBean Bremen, Mar 20 '20 at 10:24
Hm, strange. Maybe I'll have a look later tonight, out of interest. — MrBean Bremen, Mar 20 '20 at 12:01
Tested with Ubuntu 18.04 (don't have 16.04) and it still works. Can you show the complete stacktrace? Maybe you have some package installed which interferes with pickle? I'm out of ideas... — MrBean Bremen, Mar 20 '20 at 19:04

score 1 · Answer 1 · answered Mar 23 '20 at 10:25

1

Found it !

It was my version of mock that was outdated ... went from 2.0.0 to 3.0.5 and now it works fine. Thanks !

answered Mar 23 '20 at 10:25

luc.soret

21
4

score 0 · Answer 2 · answered Nov 27 '20 at 13:55

0

For python3, change __builtin__.open to builtins.open and it works also :)

answered Nov 27 '20 at 13:55

Lücks

3,806
2
40
54

How to use mock_open with pickle.load

2 Answers2