How to mock open differently depending on the parameters passed to open()

Question

My question is how to mock open in python, such that it reacts differently depending on the argument open() is called with. These are some different scenario's that should be possible:

• open a mocked file; read preset contents, the basic scenario.
• open two mocked files and have them give back different values for the read() method. The order in which the files are opened/read from should not influence the results.
• Furthermore, if I call open('actual_file.txt') to open an actual file, I want the actual file to be opened, and not a magic mock with mocked behavior. Or if I just don't want the access to a certain file mocked, but I do want other files to be mocked, this should be possible.

I know about this question: Python mock builtin 'open' in a class using two different files. But that answer only partially answers up to the second requirement. The part about order independent results is not included and it does not specify how to mock only some calls, and allow other calls to go through to the actual files (default behavior).

Answer 1

A bit late, but I just recently happened upon the same need, so I'd like to share my solution, based upon this answer from the referred-to question:

import pytest
from unittest.mock import mock_open
from functools import partial
from pathlib import Path


mock_file_data = {
    "file1.txt": "some text 1",
    "file2.txt": "some text 2",
    # ... and so on ...
}


do_not_mock: {
    # If you need exact match (see note in mocked_file(),
    # you should replace these with the correct Path() invocations
    "notmocked1.txt",
    "notmocked2.txt",
    # ... and so on ...
}


# Ref: https://stackoverflow.com/a/38618056/149900
def mocked_file(m, fn, *args, **kwargs):
    m.opened_file = Path(fn)
    fn = Path(fn).name  # If you need exact path match, remove this line
    if fn in do_not_mock:
        return open(fn, *args, **kwargs)
    if fn not in mock_file_data:
        raise FileNotFoundError
    data = mock_file_data[fn]
    file_obj = mock_open(read_data=data).return_value
    file_obj.__iter__.return_value = data.splitlines(True)
    return file_obj


def assert_opened(m, fn):
    fn = Path(fn)
    assert m.opened_file == fn


@pytest.fixture()
def mocked_open(mocker):
    m = mocker.patch("builtins.open")
    m.side_effect = partial(mocked_file, m)
    m.assert_opened = partial(assert_opened, m)
    return m


def test_something(mocked_open):
    ...
    # Something that should NOT invoke open()
    mocked_open.assert_not_called()

    ...
    # Something that SHOULD invoke open()
    mocked_open.assert_called_once()
    mocked_open.assert_opened("file1.txt")
    # Depends on how the tested unit handle "naked" filenames,
    # you might have to change the arg to:
    #   Path.cwd() / "file1.txt"

    # ... and so on ...

Do note that (1) I am using Python 3, and (2) I am using pytest.

Answer 2

This can be done by following the approach in the other question's accepted answer (Python mock builtin 'open' in a class using two different files) with a few alterations.

First off. Instead of just specifying a side_effect that can be popped. We need to make sure the side_effect can return the correct mocked_file depending on the parameters used with the open call.

Then if the file we wish to open is not among the files we wish to mock, we instead return the original open() of the file instead of any mocked behavior.

The code below demonstrates how this can be achieved in a clean, repeatable way. I for instance have this code inside of a file that provides some utility functions to make testing easier.

from mock import MagicMock
import __builtin__
from mock import patch
import sys

# Reference to the original open function.
g__test_utils__original_open = open
g__test_utils__file_spec = None

def create_file_mock(read_data):

    # Create file_spec such as in mock.mock_open
    global g__test_utils__file_spec
    if g__test_utils__file_spec is None:
        # set on first use
        if sys.version_info[0] == 3:
            import _io
            g__test_utils__file_spec = list(set(dir(_io.TextIOWrapper)).union(set(dir(_io.BytesIO))))
        else:
            g__test_utils__file_spec = file

    file_handle = MagicMock(spec=g__test_utils__file_spec)
    file_handle.write.return_value = None
    file_handle.__enter__.return_value = file_handle
    file_handle.read.return_value = read_data
    return file_handle

def flexible_mock_open(file_map):
    def flexible_side_effect(file_name):
        if file_name in file_map:
            return file_map[file_name]
        else:
            global g__test_utils__original_open
            return g__test_utils__original_open(file_name)

    global g__test_utils__original_open
    return_value = MagicMock(name='open', spec=g__test_utils__original_open)
    return_value.side_effect = flexible_side_effect
    return return_value

if __name__ == "__main__":
    a_mock = create_file_mock(read_data="a mock - content")
    b_mock = create_file_mock(read_data="b mock - different content")
    mocked_files = {
        'a' : a_mock,
        'b' : b_mock,
    }
    with patch.object(__builtin__, 'open', flexible_mock_open(mocked_files)):
        with open('a') as file_handle:
            print file_handle.read() # prints a mock - content

        with open('b') as file_handle:
            print file_handle.read() # prints b mock - different content

        with open('actual_file.txt') as file_handle:
            print file_handle.read() # prints actual file contents

This borrows some code straight from the mock.py (python 2.7) for the creating of the file_spec.

side note: if there's any body that can help me in how to hide these globals if possible, that'd be very helpful.