4

I was just wondering what the proper way to pass an argument to a custom task decorator would be. For example, I found out I can subclass a celery task as follows:

class MyTask(celery.Task):
    def __init__(self):
        # some custom stuff here
        super(MyTask, self).__init__()

    def __call__(self, *args, **kwargs):
        # do some custom stuff here
        res = self.run(*args, **kwargs)
        # do some more stuff here
        return res

and use it as below:

@MyTask
def add(x, y):
    return x + y

but I would like to be able to pass an argument to this task and have it behave differently based on the argument (or equivalently, based on the function it is decorating). There are two methods I can think of doing this. One is by passing an additional kwarg to celery's task wrapper and specifying the base explicitly, such as

@celery.task(base=MyTask, foo="bar")
def add(x, y):
    return x + y

which I can access in my custom task as self.foo, but this feels a bit like cheating to me. The other way would be to inspect self.task, and change the behaviour based on its value, but this also seems like overkill. Ideally I would like to pass the kwarg directly to the custom task class,

@MyTask(foo="bar")
def add(x, y):
    return x + y

but of course this creates an instance of MyTask, which we do not want and will not work anyhow.

Any suggestions on the proper way of doing this?

Shaun
  • 480
  • 6
  • 10

1 Answers1

1

You can add use a class-member instead of an instance-member. Therefore, you would define your arguments outside of __init__ in MyTask as shown below. You can then use this class as the base class for your celery task and use these new class-members as arguments of your custom task.

Note: You cannot pass them in the __init__ unfortunately as you would need to instantiate MyTask at decoration time.

class MyTask(celery.Task):
    foo = "default"

    def __init__(self):
        # some custom stuff here
        super(MyTask, self).__init__()

    def __call__(self, *args, **kwargs):
        # do some custom stuff here
        print(self.foo)
        res = self.run(*args, **kwargs)
        # do some more stuff here
        return res

You can then use:

@celery.task(base=MyTask, foo="bar")
def add(x, y):
    return x + y
nbeuchat
  • 6,575
  • 5
  • 36
  • 50
  • This doesn't achieve what I wanted though. You can already access "foo" within the base class using the `@celery.task(base=MyTask, foo="bar")` decorator. – Shaun Mar 31 '19 at 16:37
  • @Shaun could you specify what you mean by "change the behavior based on its value". In my answer, you can do something before or after (or even change/remove/add args or kwargs) in the `__call__` function. – nbeuchat Apr 01 '19 at 15:08
  • Btw, the idea of adding `foo = "default"` in `MyTask` is just to have a default value for it in `MyTask`. The most important part is actually what you do with it in `__call__` – nbeuchat Apr 01 '19 at 15:30
  • I outlined the method you posted in my question. But I was looking for a more direct way of passing it to the custom class. – Shaun Apr 01 '19 at 16:43