Template method invocation order in python

Question

What would be the simplest way to unit-test run function? Here I'm not interested in what fun* functions do, but only if the order of their invocation is correct.

from mock import Mock

(many, arguments, four, fun) = ('many', 'arguments', 'four', 'fun') 

class TemplateMethod(object):

    def fun1(self, many, parameters, go, here):
        raise NotImplementedError()

    @classmethod
    def fun2(cls, many, parameters, go, here):
        pass

    @staticmethod
    def fun3(many, parameters, go, here):
        pass

    def run(self):
        result1 = self.fun1(many, arguments, four, fun)
        result1.do_something()

        self.fun2(many, arguments, four, fun)
        self.fun3(many, arguments, four, fun)

The only requirement for the solution is non-intrusiveness towards the class under test.

SOLUTION:

This is somewhat of a draft, actually, and this simple class could be fixed and extended in so many ways that I don't care even to think about it. The point is that you can now simply record all the invocations of all the methods in a template method. You can also specify a list of object that shouldn't be mocked by the class (i.e. the function you are recording invocations for).

Special thanks go to @Damian Schenkelman, who gave me some important pointers.

class MockingInvocationRecorder(object):

    FUNCTION_TYPES = ('function', 'classmethod', 'staticmethod')

    def __init__(self, obj, dont_mock_list):
        self._invocation_list = []

        name_list = [exc.__name__ for exc in dont_mock_list]
        self._wrap_memfuns(obj, name_list)

    @property
    def invocations(self):
        return tuple(self._invocation_list)

    def _wrap_single(self, memfun, exclude_list):
        def wrapper(*args, **kwargs):
            self._invocation_list.append(memfun.__name__)

            if memfun.__name__ in exclude_list:
                return memfun(*args, **kwargs)
            else:
                return Mock()

        return wrapper

    def _wrap_memfuns(self, obj, exclude_list):
        for (mem_name, mem) in type(obj).__dict__.iteritems():
            if type(mem).__name__ in self.FUNCTION_TYPES:
                wrapper = self._wrap_single(getattr(obj, mem_name), exclude_list)
                setattr(obj, mem_name, wrapper)

You can now test invocation order by something like this:

tm = TemplateMethod()
ir = MockingInvocationRecorder(tm, [tm.run])

tm.run()

print ir.invocations => ('run', 'fun1', 'fun2', 'fun3')

You can reference a member by class name:

ir = MockingInvocationRecorder(tm, [TemplateMethod.run])

Just be careful that you enlist all the methods that shouldn't be mocked away.

This strikes me as a suspicious design. Why would you need to check that the order of the functions is correct? What if some change to your program later requires the functions to be executed in a different order? You'll have to basically copy the change over to the unit test. It seems like a violation of the *Don't Repeat Yourself* principle to hard-code in the *order* of the function calls as part of the test. There must be a better, more abstract way to test the output of `run` without caring how it goes about getting to the output, no? — ely, Sep 22 '12 at 18:30
The whole point is to make sure that the algorithm steps within the template method are actually executed in a specific order. In my actual code, I'm enforcing authentication and authorization first. As for the design, this is just a POC. I don't like the idea of having the call_appender class, but since I'm a newbie in Python, I don't see any simpler solution. I could extend current solution with reflection, but I was kind a hoping that someone will propose better design. (I'm not asking for complete solution, just ideas.) — LavaScornedOven, Sep 22 '12 at 19:35

score 3 · Accepted Answer · answered Sep 22 '12 at 16:22

3

Monkeypatch fun1, fun2, fun3 functions and use a list to keep track of the call order. After that, assert on the list.

answered Sep 22 '12 at 16:22

Damian Schenkelman

3,505
1
15
19

How would I use a list to record the calls? – LavaScornedOven Sep 22 '12 at 16:56
Declare a list in the test method, and in each monkeypatching lambda you can simply add 1 to the list for fun1, 2 for fun2, etc... – Damian Schenkelman Sep 23 '12 at 00:10
I already tried lambdas, and the problem is that I don't know how to pass them a list except as an argument. Besides. lambdas would work for mocking only if the return value of mocked functions is not used within `run` method (as in my example, but I am aiming at some generality here), because return value of `append` is None, IIRC. – LavaScornedOven Sep 23 '12 at 00:34
A lambda (or closure) has access to variables defined in the scope were it was defined, thus you don't need to pass the list as a parameter. For example: fun1 = lambda: invocations.append("fun1") can be used, having declared invocations as: invocations = [] within the unit test. – Damian Schenkelman Sep 23 '12 at 02:48
OK, lambdas are closures in python... Didn't know that. Thnx. Now I just need to find a hack which will allow me to execute two statements in lambda. In updated version of code, you can see that do_something is called on return value of one of the functions. Is conditional expression in lambda way to achieve that? Something like this: `lambda *argv, **kwargv: SomeClass() if invocations.append('fun1') is None else SomeClass()`? Is there some other way to achieve the same effect, i.e. return some specific value and append to invocations? – LavaScornedOven Sep 23 '12 at 13:27
To execute more than one statement, define a function that receives a list. In the lambda, invoke the function passing the invocations list as parameter, and use append as before. In the function you can use as much code as you want to. – Damian Schenkelman Sep 23 '12 at 14:03
OK, thnx for everything. I'll integrate this to my solution. – LavaScornedOven Sep 23 '12 at 14:58
@Vedran: Glad to help. In general, if an answer is useful you should mark it as "Answer". That way other people that read the thread can find the answer fast and you also give credit to the person who provideded the useful answer. – Damian Schenkelman Sep 23 '12 at 15:15
let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/17021/discussion-between-damian-schenkelman-and-vedran) – Damian Schenkelman Sep 23 '12 at 15:16
let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/17036/discussion-between-damian-schenkelman-and-vedran) – Damian Schenkelman Sep 23 '12 at 21:50

score 2 · Answer 2 · answered Sep 22 '12 at 20:43

2

Making use of __call__ could be helpful here.

class Function1(object):
    def __call__(self, params, logger=None):
        if logger is not None:
            logger.append(self.__repr__())
        return self.function1(params) 
        # Or no return if only side effects are needed

    def __init__(self, ...):
        #...

    def __repr__(self):
        # Some string representation of your function.

    def function1(params):
        print "Function 1 stuff here..."

Then do something like

class TemplateMethod(object):
    def __init__(self, params):
        self.logger = []
        self.fun1 = Function1(...)
        self.fun1(params, self.logger)

This is pretty hacky; there's probably some ways to clean up what I'm getting at, but encapsulating the functions in classes and then using __call__ is a good way to go.

answered Sep 22 '12 at 20:43

ely

74,674
34
147
228

I like the idea, but the problem with this approach is that you're modifying class under test to accommodate testing, and not the other way around. I was looking for a non-intrusive solution. – LavaScornedOven Sep 22 '12 at 20:53
It's true, but if you give `TemplateMethod` a logger, then in your unit test code, you can pull in the specifications for what that logger is supposed to look like at testing time, with no need for the test code to know anything about the names of the functions or what order they are supposed to be called in. That can all be metadata, which itself could be extracted from the source code, specs, or docs for `TemplateMethod`. I will be interested in other solutions that extract `TemplateMethod`'s function calls directly though. – ely Sep 22 '12 at 20:56
1

Yup, I agree. BTW, your approach could blend in much smoother with decorators. – LavaScornedOven Sep 22 '12 at 21:10

Template method invocation order in python

2 Answers2