75

Suppose there are two packages in a project: some_package and another_package.

# some_package/foo.py:
def bar():
    print('hello')

# another_package/function.py
from some_package.foo import bar

def call_bar():
    # ... code ...
    bar()
    # ... code ...

I want to test another_package.function.call_bar mocking out some_package.foo.bar because it has some network I/O I want to avoid.

Here is a test:

# tests/test_bar.py
from another_package.function import call_bar

def test_bar(monkeypatch):
    monkeypatch.setattr('some_package.foo.bar', lambda: print('patched'))
    call_bar()
    assert True

To my surprise it outputs hello instead of patched. I tried to debug this thing, putting an IPDB breakpoint in the test. When I manually import some_package.foo.bar after the breakpoint and call bar() I get patched.

On my real project the situation is even more interesting. If I invoke pytest in the project root my function isn't patched, but when I specify tests/test_bar.py as an argument - it works.

As far as I understand it has something to do with the from some_package.foo import bar statement. If it's being executed before monkeypatching is happening then patching fails. But on the condensed test setup from the example above patching does not work in both cases.

And why does it work in IPDB REPL after hitting a breakpoint?

Nathaniel Jones
  • 939
  • 1
  • 14
  • 25
Glueon
  • 3,727
  • 4
  • 23
  • 32

5 Answers5

89

While Ronny's answer works it forces you to change application code. In general you should not do this for the sake of testing.

Instead you can explicitly patch the object in the second package. This is mentioned in the docs for the unittest module.

monkeypatch.setattr('another_package.bar', lambda: print('patched'))
Alex
  • 18,484
  • 8
  • 60
  • 80
  • 11
    This is clearly the cleaner way of monkeypatching an import. – Floran Gmehlin May 05 '18 at 09:50
  • The link is to the wrong docs as far as I can tell. It should be to [the pytest monkeypatch docs](https://docs.pytest.org/en/latest/reference.html#_pytest.monkeypatch.MonkeyPatch). – LondonRob Mar 09 '20 at 18:26
  • @LondonRob the unit-test docs have an explanation for why patching is a good idea – Alex Mar 10 '20 at 14:09
  • 1
    Let me share an example when changing a code might be a good idea: You have some widely used functions and write some component/integration tests. You need to monkeypatch each model there it is used. `monkeypatch.setattr('a.val', None); monkeypatch.setattr('b.val', None)` – Cjkjvfnby Jan 14 '21 at 21:21
  • @Cjkjvfnby I see your point and I'm literally facing the exact issue you describe. However, personally I feel that the point still stands and it would be better to have multiple `monkeypatch` statements in the test vs modifying code structure just for the sake of testing. An exception to this would be if writing the test actually highlighted that the code wasn't written well, in which case it would justify a rewrite. – LondonAppDev Jul 15 '21 at 08:54
  • 1
    You just saved my life. I was patching the original function and not the imported one. I don't know how many times I've had this issue.. – Akaisteph7 May 19 '23 at 01:10
42

As Alex said, you shouldn't rewrite your code for your tests. The gotcha I ran into is which path to patch.

Given the code:

app/handlers/tasks.py

from auth.service import check_user

def handle_tasks_create(request):
  check_user(request.get('user_id'))
  create_task(request.body)
  return {'status': 'success'}

Your first instinct to monkeypatch check_user, like this:

monkeypatch.setattr('auth.service.check_user', lambda x: return None)

But what you want to do is patch the instance in tasks.py. Likely this is what you want:

monkeypatch.setattr('app.handlers.tasks.check_user', lambda x: return None)

While the answers given are already good, I hope this brings more complete context.

alairock
  • 1,826
  • 1
  • 21
  • 28
27

Named importation creates a new name for the object. If you then replace the old name for the object the new name is unaffected.

Import the module and use module.bar instead. That will always use the current object.

import module 

def func_under_test():
  module.foo()

def test_func():
   monkeypatch.setattr(...)
   func_under_test
Neuron
  • 5,141
  • 5
  • 38
  • 59
  • 17
    This is among the worst gotchas with pytest — but thanks for explaining it. – Brighid McDonnell Aug 05 '16 at 04:06
  • 8
    When you say "use module.bar", can you provide a code example? I have tried `monkeypatch.setattr(module, 'bar', mock_obj)` and a few other incantations without success. – skolsuper Aug 18 '16 at 05:18
  • 1
    Thank you for your answer. Is there something trick to be sure that any import will use mocked object. Because for now it is dangerous if, for example I'm trying to mock orm object – Stavinsky Feb 02 '17 at 21:03
  • 5
    this is a property of the python language and there is no trick –  Feb 21 '17 at 15:56
2

correct answer to OP's question:

monkeypatch.setattr('another_package.function.bar', lambda: print('patched'))
Felix Liu
  • 199
  • 2
  • 9
2

Another possible reason your function may not be getting patched is if your code is using multiprocessing.

On macOS the default start method for a new process has changed from fork to spawn. If spawn is used, a brand new Python interpreter process is started, ignoring your recently patched function.

Fix: Set the default start method to fork.

import multiprocessing

multiprocessing.set_start_method('fork', force=True)

You can add this snippet to conftest.py inside your tests/ folder.

Mastergalen
  • 4,289
  • 3
  • 31
  • 35