9

For example, I have some module(foo.py) with next code:

import requests

def get_ip():
    return requests.get('http://jsonip.com/').content

And module bar.py with similiar code:

import requests

def get_fb():
    return requests.get('https://fb.com/').content

I just can't understand why next happens:

from mock import patch

from foo import get_ip
from bar import get_fb

with patch('foo.requests.get'):
    print(get_ip())
    print(get_fb())

They are two mocked: <MagicMock name='get().content' id='4352254472'> <MagicMock name='get().content' id='4352254472'> It is seemed to patch only foo.get_ip method due to with patch('foo.requests.get'), but it is not. I know that I can just get bar.get_fb calling out of with scope, but there are cases where I just run in context manager one method that calls many other, and I want to patch requests only in one module. Is there any way to solve this? Without changing imports in module

hasam
  • 155
  • 1
  • 6

2 Answers2

5

The two locations foo.requests.get and bar.requests.get refer to the same object, so mock it in one place and you mock it in the other.

Imagine how you might implement patch. You have to find where the symbol is located and replace the symbol with the mock object. On exit from the with context you will need to restore the original value of the symbol. Something like (untested):

class patch(object):
    def __init__(self, symbol):
        # separate path to container from name being mocked
        parts = symbol.split('.')
        self.path = '.'.join(parts[:-1]
        self.name = parts[-1]
    def __enter__(self):
        self.container = ... lookup object referred to by self.path ...
        self.save = getattr(self.container, name)
        setattr(self.container, name, MagicMock())
    def __exit__(self):
        setattr(self.container, name, self.save)

So your problem is that the you are mocking the object in the request module, which you then are referring to from both foo and bar.

Following @elethan's suggestion, you could mock the requests module in foo, and even provide side effects on the get method:

from unittest import mock
import requests

from foo import get_ip
from bar import get_fb

def fake_get(*args, **kw):
    print("calling get with", args, kw)
    return mock.DEFAULT

replacement = mock.MagicMock(requests)
replacement.get = mock.Mock(requests.get, side_effect=fake_get, wraps=requests.get)
with mock.patch('foo.requests', new=replacement):
    print(get_ip())
    print(get_fb())

A more direct solution is to vary your code so that foo and bar pull the reference to get directly into their name space.

foo.py:

from requests import get

def get_ip():
    return get('http://jsonip.com/').content

bar.py:

from requests import get

def get_ip():
    return get('https://fb.com/').content

main.py:

from mock import patch

from foo import get_ip
from bar import get_fb

with patch('foo.get'):
    print(get_ip())
    print(get_fb())

producing:

<MagicMock name='get().content' id='4350500992'>
b'<!DOCTYPE html>\n<html lang="en" id="facebook" ...

Updated with a more complete explanation, and with the better solution (2016-10-15)

Note: added wraps=requests.get to call the underlying function after side effect.

Neapolitan
  • 2,101
  • 9
  • 21
  • As far as I can tell, you can get the same effect by keeping foo.py and bar.py the same, and mocking `foo.requests` instead of `foo.requests.get` – elethan Oct 15 '16 at 18:16
  • Thanks, but I also know this solution and mentioned in post that `Without changing imports in module`. I hope there is a solution that does not require changing script imports – hasam Oct 15 '16 at 18:20
  • @elethan this works, thnx ;) But why when mocking `foo.requests` it mocks only `requests` in `foo.py`, but when mocking `foo.requests.get`, it mocks `foo.py` and `bar.py`. Can you write a full answer why it happens? – hasam Oct 15 '16 at 18:23
  • 1
    @hasam I think this is because when you mock `foo.requests` it is mocking the module object that has already been imported into `foo`, and so doesn't affect the one that will be imported in `bar`. However, when you mock `foo.requests.get` it will look up the `requests` object imported in `foo`, then look up `get` in the original module, and mock that, so when `bar` imports `requests` it is getting a mocked `get` method. Does that make sense? – elethan Oct 15 '16 at 18:28
  • @elethan yes, thanks. I think you should write an answer to my question and I will accept it, because that was exactly what I needed – hasam Oct 15 '16 at 18:31
  • @hasam done. Although this answer is basically a different route to the same solution. Mine is just slightly different. This answer at least deserves an upvote if you don't want to accept it. – elethan Oct 15 '16 at 18:45
1

Not to steal @Neapolitan's thunder, but another option would be to simply mock foo.requests instead of foo.requests.get:

with patch('foo.requests'):
    print(get_ip())
    print(get_fb())

I think the reason why both methods get mocked in your case is that, since requests.get is not explicitly imported in foo.py, mock will have to look up the method in the requests module and mock it there, rather than mocking it in the requests object already imported into foo, so that when bar later imports requests and accesses requests.get it is geting the mocked version. However, if you patch foo.requests instead, you are just patching the module object already imported into foo, and the original requests module will not be affected.

Although not particularly helpful for this particular problem, this article is very useful for understanding the subtleties of patch

elethan
  • 16,408
  • 8
  • 64
  • 87
  • is there any way to put a `side_effect` on `requests.get` method with this approach? – hasam Oct 15 '16 at 19:10
  • @hasam Yes, as long as you `import foo` in your test module, you should be able to do `foo.requests.get.side_effect = whatever_side_effect`, since `foo.requests.get` will be a `Mock` object. – elethan Oct 16 '16 at 02:20