How to use async function inside celery task?

Question

I've found some similar questions but couldn't find what I want.
I have async function which is I want to use it inside my celery task but cannot call it with await inside task. Is there any way to do it?

db.py

async def select_users():
    sql = "SELECT * FROM Users WHERE "
    sql, parameters = self.format_args(sql, parameters=kwargs)
    return await self.execute(sql, *parameters, fetchrow=True)

tasks.py

from .celery import app
import db

@app.task
def update_credits():
    users = db.select_users()  #here I should call func with 'await'
    print(users)

score 1 · Answer 1 · answered Feb 13 '23 at 15:05

You can do something like the following:

Define a decorator to apply async to sync. This requires asgiref.

from functools import wraps
from celery import Celery, Task
from typing import Any, Callable, Coroutine, ParamSpec, TypeVar
from asgiref import sync


_P = ParamSpec("_P")
_R = TypeVar("_R")


def async_task(app: Celery, *args: Any, **kwargs: Any):
    def _decorator(func: Callable[_P, Coroutine[Any, Any, _R]]) -> Task:
        sync_call = sync.AsyncToSync(func)

        @app.task(*args, **kwargs)
        @wraps(func)
        def _decorated(*args: _P.args, **kwargs: _P.kwargs) -> _R:
            return sync_call(*args, **kwargs)

        return _decorated

    return _decorator

Apply the decorator to your async function:

import asyncio
import celery
from celery.utils.log import get_task_logger

from ..celery import app
from .async_task import async_task


@async_task(app, bind=True)
async def test_task(self: celery.Task):
    await asyncio.sleep(1.0)
    return "world"


@app.task(ignore_result=True)
def say_hello(who: str):
    print(f"Hello {who}")

Hope this help you.

score 0 · Answer 2 · answered Jun 07 '22 at 15:03

0

You can use the Event Loop method run_until_complete:

import asyncio
from .celery import app
import db

@app.task
def update_credits():
    loop = asyncio.get_event_loop()
    users = loop.run_until_complete(db.select_users())
    print(users)

But if you do not await the db.select_users() coroutine inside an async def your code is not getting any improve by using asyncio.

answered Jun 07 '22 at 15:03

svex99

518
4
10

Getting warning `RuntimeWarning: coroutine 'update_credits' was never awaited result = (True, prepare_result(fun(*args, **kwargs)))` – mirodil Jun 07 '22 at 15:32
I guess that error is not related to the question. Did you change `update_credits` to an async def? – svex99 Jun 07 '22 at 15:37
yes I changed to async – mirodil Jun 07 '22 at 16:08
In that case, you have to `await` it in the context you are calling it. – svex99 Jun 07 '22 at 16:36

How to use async function inside celery task?

2 Answers2