How does MagicMock avoid throwing AttributeError when a random method is called?

Question

In Python, if you call a method that doesn't exist it throws an AttributeError. Ex

>>> class A:
...     def yo(self):
...             print(1)
... 
>>> a = A()
>>> a.yo()
1
>>> a.hello()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'A' object has no attribute 'hello'

In the below code, MagicMock class doesn't have a function named hello or no patch was created for a method hello. Still below code doesn't throw an AttributeError

>>> from unittest.mock import MagicMock 
>>> obj = MagicMock()
>>> obj.hello()
<MagicMock name='mock.hello()' id='4408758568'>

How is MagicMock able to do this? How can I create a class that can perform an action when any method (which might not be defined) is called on it?

I thought there was a special method that got called every time a function is called. And MagicMock overrides that. — 2ank3th, Oct 24 '18 at 22:24
The mock documentation refers to this repeatedly; you need to *spec* the mock if you want to see attribute errors. — jonrsharpe, Oct 24 '18 at 22:25
I understand that with autospec=True, it will throw AttributeError. My question is how is MagicMock able to return mock for any method that get's called open it's object. — 2ank3th, Oct 24 '18 at 22:27

wim · Accepted Answer · 2018-10-25T01:01:29.683

The Python datamodel documents a hook, __getattr__, which shall be called when attribute access fails to resolve in the usual ways. Mocks use it to return a new mock instance - i.e. mocks define unknown attributes as factories.

Reproducing mock's implementation in a simpler way, you would just turn __getattr__ and __call__ into factory functions:

class M:
    def __call__(self):
        return M()
    def __getattr__(self, name):
        return M()

Example usage:

>>> mock = M()
>>> mock.potato
<__main__.M at 0xdeadbeef>
>>> mock.potato()
<__main__.M at 0xcafef00d>

How is MagicMock able to do this?

This part is not specific to MagicMock, an ordinary Mock will do the same (the "magic" in the name is just referring to additional features allowing better mocking of magic methods). MagicMock inherits such behavior from one of the base classes:

>>> MagicMock.mro()
[unittest.mock.MagicMock,
 unittest.mock.MagicMixin,
 unittest.mock.Mock,
 unittest.mock.CallableMixin,
 unittest.mock.NonCallableMock,  # <--- this one overrides __getattr__!
 unittest.mock.Base,
 object]

How can I create a class that can perform an action when any method (which might not be defined) is called on it?

It depends if you want to be in front of, or behind, a normal attribute access. If you want to get in front, you should define __getattribute__, it's called unconditionally to implement attribute accesses before searching the class/instance namespaces. However, if you want to take a lower precedence to normal attributes (i.e. those living in the object __dict__) and to descriptors, then you should define __getattr__ as previously discussed.

Is there a way that Mock or MagicMock itself support pre/post behavior for any method call on the object its mocking? I have a function that holds the responses of my mock object's calls and I'd like any call on that mock to complete with the result of that function. — LeanMan, Oct 16 '21 at 23:29

score 1 · Answer 2 · answered Oct 24 '18 at 22:36

I don't actually know how specifically MagicMock works (I've never used it, but I've Heard Good Things), but this part of the behaviour can be replicated (along with probably multiple other possible solutions) by hijacking __getattr__ in a way that it returns a callable that creates a new mock instance when called:

class MM:
    def __init__(self, name=None):
        # store a name, TODO: random id, etc.
        self.name = name

    def __repr__(self):
        # make it pretty
        if self.name:
            r = f'<MM name={self.name}>'
        else:
            r = f'<MM>'
        return r

    def __getattr__(self, attrname):
        # we want a factory for a mock instance with a name corresponding to attrname
        def magicattr():
            return MM(name=f"'mock.{attrname}()'")
        return magicattr

When executed, we see the following:

>>> MM()
<MM>
>>> MM().hello()
<MM name='mock.hello()'>

I didn't go overboard with defining an id and whatnot, but the basic trick can be seen on the above stripped-down example.

The way the above works is that accessing .hello or any other attribute goes through our custom __getattr__ which gives us a chance to generate a fake (mocked) method on the fly, with whatever properties we want to. As I understand one of the many benefits of MagicMock is exactly that we don't have to worry about AttributeErrors being thrown by default, it just works.

@2ank3th I suggest changing the accept to wim's answer. I only posted one because people more familiar with the subject didn't seem to bite. — Andras Deak -- Слава Україні, Oct 25 '18 at 00:13

How does MagicMock avoid throwing AttributeError when a random method is called?

2 Answers2