Mocking a file containing JSON data with mock.patch and mock_open

Question

I'm trying to test a method that requires the use of json.load in Python 3.6. And after several attempts, I tried running the test "normally" (with the usual unittest.main() from the CLI), and in the iPython REPL.

Having the following function (simplified for purpose of the example)

def load_metadata(name):
    with open("{}.json".format(name)) as fh:
        return json.load(fh)

with the following test:

class test_loading_metadata(unittest2.TestCase):
    @patch('builtins.open', new_callable=mock_open(read_data='{"disabled":True}'))
    def test_load_metadata_with_disabled(self, filemock):
        result = load_metadata("john")
        self.assertEqual(result,{"disabled":True})
        filemock.assert_called_with("john.json")

The result of the execution of the test file, yields a heart breaking:

TypeError: the JSON object must be str, bytes or bytearray, not 'MagicMock'

While executing the same thing in the command line, gives a successful result.

I tried in several ways (patching with with, as decorator), but the only thing that I can think of, is the unittest library itself, and whatever it might be doing to interfere with mock and patch.

Also checked versions of python in the virtualenv and ipython, the versions of json library.

I would like to know why what looks like the same code, works in one place and doesn't work in the other. Or at least a pointer in the right direction to understand why this could be happening.

Your title and your question don't seem to be related all that much otherwise. What function are you talking about? Is the class the unittest? The issue here has nothing to do with methods or functions otherwise. — Martijn Pieters, Mar 04 '19 at 21:10
If you had a test *function* that *worked*, then you forgot to share that, and you were probably not using `new_callable` or had wrapped the callable in a lambda. It would not have been the exact same `patch(...)` configuration. — Martijn Pieters, Mar 04 '19 at 21:13
I forgot to share the test _function_. Sorry for that. I checked it again, and you are right, it was a slightly different patch config. Thanks for the corrections to the question as well. I'm so tired I'm not even writing straight. — Wakaru44, Mar 04 '19 at 21:42
Time to get yourself some sleep then, you'll be much more productive afterwards. :-) — Martijn Pieters, Mar 04 '19 at 21:57

score 5 · Accepted Answer · answered Mar 04 '19 at 20:53

json.load() simply calls fh.read(), but fh is not a mock_open() object. It's a mock_open()() object, because new_callable is called before patching to create the replacement object:

>>> from unittest.mock import patch, mock_open
>>> with patch('builtins.open', new_callable=mock_open(read_data='{"disabled":True}')) as filemock:
...     with open("john.json") as fh:
...         print(fh.read())
...
<MagicMock name='open()().__enter__().read()' id='4420799600'>

Don't use new_callable, you don't want your mock_open() object to be called! Just pass it in as the new argument to @patch() (this is also the second positional argument, so you can leave off the new= here):

@patch('builtins.open', mock_open(read_data='{"disabled":True}'))
def test_load_metadata_with_disabled(self, filemock):

at which point you can call .read() on it when used as an open() function:

>>> with patch('builtins.open', mock_open(read_data='{"disabled":True}')) as filemock:
...     with open("john.json") as fh:
...         print(fh.read())
...
{"disabled":True}

The new argument is the object that'll replace the original when patching. If left to the default, new_callable() is used instead. You don't want new_callable() here.

Mocking a file containing JSON data with mock.patch and mock_open

1 Answers1