48

I'm trying to use the python mock library to patch a Celery task that is run when a model is saved in my django app, to see that it's being called correctly.

Basically, the task is defined inside myapp.tasks, and is imported at the top of my models.py-file like so:

from .tasks import mytask

...and then runs on save() inside the model using mytask.delay(foo, bar). So far so good - works out fine when I'm actually running Celeryd etc.

I want to construct a unit test that mocks the task, just to check that it gets called with the correct arguments, and doesn't actually try to run the Celery task ever.

So in the test file, I've got something like this inside of a standard TestCase:

from mock import patch # at the top of the file

# ...then later
def test_celery_task(self):
    with patch('myapp.models.mytask.delay') as mock_task:
        # ...create an instance of the model and save it etc
        self.assertTrue(mock_task.called)

...but it never gets called/is always false. I've tried various incarnations (patching myapp.models.mytask instead, and checking if mock_task.delay was called instead. I've gathered from the mock docs that the import path is crucial, and googling tells me that it should be the path as it is seen inside the module under tests (which would be myapp.models.mytask.delay rather than myapp.tasks.mytask.delay, if I understand it correctly).

Where am I going wrong here? Is there some specific difficulties in patching Celery tasks? Could I patch celery.task (which is used as a decorator to mytask) instead?

Emil
  • 1,949
  • 2
  • 16
  • 25

3 Answers3

59

The issue that you are having is unrelated to the fact that this is a Celery task. You just happen to be patching the wrong thing. ;)

Specifically, you need to find out which view or other file is importing "mytask" and patch it over there, so the relevant line would look like this:

with patch('myapp.myview.mytask.delay') as mock_task:

There is some more flavor to this here:

http://www.voidspace.org.uk/python/mock/patch.html#where-to-patch

Thanos Diacakis
  • 948
  • 9
  • 10
  • Cheers! I've yet to try it out (project is dormant right now) but will try it out soon and mark this as answered. I seem to recall having tried a bunch of variations on the theme you're suggesting, but it's fully possible my blood sugar was low at the time... :-) – Emil Dec 09 '13 at 16:30
  • 1
    Actually, I'm doing this pretty much exactly as you suggest, as exemplified in the question code... Cannot get it to work. Oh well. – Emil Feb 23 '14 at 15:44
  • The question is patching the model. That smells wrong as I suspect you are not using "delay" in the model, but somewhere else - possibly a view, hence my patch code (above) is slightly different. – Thanos Diacakis Feb 23 '14 at 22:53
  • Well, I'm calling `mytask.delay() ` inside of the overridden `save()` method on the model, actually. (I'm using the task to send some data from the model to an external system). Would that make a difference in how the patching works? – Emil Feb 24 '14 at 11:31
  • Went back and tried everything from scratch: this time it worked fine. No idea what I was doing wrong before, but I'm happy. :-) – Emil Apr 05 '14 at 14:42
  • Thanks! This was the only solution that worked for me. Other solutions like overriding some celery settings (e.g: `CELERY_ALWAYS_EAGER`, `BROKER_BACKEND`, etc) did not work. – Ed Patrick Tan Jun 06 '14 at 08:48
44

The @task decorator replaces the function with a Task object (see documentation). If you mock the task itself you'll replace the (somewhat magic) Task object with a MagicMock and it won't schedule the task at all. Instead mock the Task object's run() method, like so:

@override_settings(CELERY_TASK_ALWAYS_EAGER=True)
@patch('monitor.tasks.monitor_user.run')
def test_monitor_all(self, monitor_user):
    """
    Test monitor.all task
    """

    user = ApiUserFactory()
    tasks.monitor_all.delay()
    monitor_user.assert_called_once_with(user.key)
java-addict301
  • 3,220
  • 2
  • 25
  • 37
Danielle Madeley
  • 2,616
  • 1
  • 19
  • 26
  • 1
    There a reason you posted this answer, word for word, on two questions? – Nathan Tuggy Mar 26 '15 at 01:07
  • It's pretty useful information on this question as well as the other. They're not precisely the same question though. – Danielle Madeley Mar 26 '15 at 04:10
  • BTW if you think that an answer is is related and useful for two questions you can flag one of them as duplicate and somebody will take care of it. Post the same answer is the wrong thing to do. – Michele d'Amico Mar 26 '15 at 10:55
  • 5
    @Micheled'Amico: I disagree, the answer talks to the specific problem of it not being run if you mock the task itself which this explains. Mocking the `delay()` method, while likely to work in a specific case is actually the wrong answer because you might later change the code to use `apply_async()` or other method and suddenly your tests will break for the wrong reason. – Danielle Madeley Mar 26 '15 at 11:24
  • Ok, I read better the question and you are right. I'll fix my vote and remove my comments. – Michele d'Amico Mar 26 '15 at 11:39
  • 1
    +1 @DanielleMadeley. I originally patched delay and the tests worked great until we needed to chain them as subtasks and everything went south in a hury! – Craig Oct 17 '16 at 23:04
  • 5
    In newer versions, we need to use `CELERY_TASK_ALWAYS_EAGER` – ericls Aug 22 '17 at 13:54
  • 2
    I think this is a way better answer because it works for chained tasks as well as tasks called with `apply_async` (to provide a `countdown` parameter). – Artem Gordinsky Sep 25 '19 at 09:28
  • https://docs.celeryq.dev/en/stable/userguide/testing.html ```The eager mode enabled by the task_always_eager setting is by definition not suitable for unit tests.``` ```To test task behavior in unit tests the preferred method is mocking.``` – LuisKarlos Oct 11 '22 at 19:40
  • I used this method because I wanted to inspect the mock.call_args that were passed to the task, and it worked great! https://docs.python.org/3/library/unittest.mock.html#unittest.mock.Mock.call_args – partofthething Mar 15 '23 at 18:48
3

Just patch the celery Task method

mocker.patch("celery.app.task.Task.delay", return_value=1)
jTiKey
  • 696
  • 5
  • 13
  • Not sure of the structure of your app but mock can't find my tasks this way – boatcoder Mar 26 '22 at 00:38
  • 1
    @boatcoder depends on the version of celery you are using. You can just look in the source code where that function is located. – jTiKey Mar 27 '22 at 01:04