Sending message from Celery task to Channels

Question

Django 2.1.1, Django Channels 2.1.3, Celery 4.2.1

I've set up a task in Celery and at the end of the task, I need to send a websocket message to the client(s). However, the websocket message is never sent. There are no errors thrown, it just simply doesn't send.

I've set up a channel layer using Redis as the backend. Doing this from a normal Django view works fine. But when run in a Celery task, it sends the message to Channels and I can see that Channels does indeed run the code shown in my consumers.py code below, but the client never receives the websocket message.

tasks.py

def import_job(self):
    # (do task calculations, store in data dict)
    message = {'type': 'send_my_data',
               'data': json.dumps(thecalcs) }
    channel_layer = get_channel_layer()
    async_to_sync(channel_layer.group_send)('core-data', message)

consumers.py

class AsyncDataConsumer(AsyncWebsocketConsumer):
    async def connect(self):
        self.channel_group_name = 'core-data'

        # Join the group
        await self.channel_layer.group_add(
            self.channel_group_name,
            self.channel_name
        )
        await self.accept()

    async def disconnect(self, close_code):
        # Leave the group
        await self.channel_layer.group_discard(
            self.channel_group_name,
            self.channel_name
        )

    # Receive message from WebSocket
    async def receive(self, text_data=None, bytes_data=None):
        pass

    # Receive message from the group
    async def send_my_data(self, event):
        text = event['data']
        # Send message to WebSocket
        await self.send(text_data=text)

settings.py

CHANNEL_LAYERS = {
    'default': {
        'BACKEND': 'channels_redis.core.RedisChannelLayer',
        'CONFIG': {
            "hosts": [('127.0.0.1', 6379)],
        },
    },
}

Since there is no exception/error, I am completely at a loss as to which part of this process is failing.

Celery triggers the task? Yes
The task runs and sends a message to the channel layer? Yes
The consumer receives the message from the group and executes the send()? Yes
Client receives the a websocket message? NO

Is this a problem between Channels and Redis? Is it a problem between Channels and the client?

Not really sure but my guess is that since the process holding the websocket connection is different from the process on which you're running the task(celery worker) it has no connection to send the message — Ken4scholars, Mar 25 '19 at 14:50
Nevermind, I've read about cases of sending channel messages from celery tasks so I was wrong — Ken4scholars, Mar 25 '19 at 15:00
Please show your complete `consumers.py` or the complete Consumer class. And have you added channel to group in your `ws_connect`? Were you able to send and receive messages if not via tasks? What is `self.group_name`? Is your channel added to this group? — Sreekanth Reddy Balne, Mar 25 '19 at 16:47
@spiritsree I have updated this with the full ```consumers.py``` file. Yes, the group send works fine when executed from regular Django code. Only in Celery tasks does it not send the websocket message. I have been using this consumer and websocket setup for months with no problem. It is only when I tried using it in Celery tasks that I see a problem. — Charles Koch, Mar 25 '19 at 17:17
It's okay. Did you check the celery logs? Any errors? How are you implementing concurrency? `eventlet` or `gevent` or `prefork`? — Sreekanth Reddy Balne, Mar 25 '19 at 20:43
@spiritsree I'm using the defaiult, which is ```prefork```. No errors from Celery or Channels and following the code with a debugger I can see that steps 1 through 3 in my description above do indeed execute. It just seems like the actual websocket message never gets sent. — Charles Koch, Mar 26 '19 at 12:56

score 5 · Accepted Answer · answered Mar 26 '19 at 14:49

5

It turns out Celery was swallowing an exception in my code during the task. I need to implement more thorough logging to catch these exceptions.

answered Mar 26 '19 at 14:49

Charles Koch

151
1
8

2

Hi Charles I'm having the same problem as you. I can see inside the Redis Cli that a message is being sent somewhere on the channel layer from the Celery background task, but my consumers are not handling the event and passing it down the socket to client. In fact sometimes the socket is disconnecting itself at the point when the message gets on the channels layer. How did you sort this out at your end? Cheers Marco – Marco Polo May 20 '19 at 12:05
@MarcoPolo Hi, I'm having the same issue as your's. How did you sort out the error? – laplace May 15 '20 at 12:22
Hi gii96, I never managed to sort this out sorry. Things became very complicated using Celery with channels. I ended up backing away from it. – Marco Polo May 17 '20 at 09:18
Hi gii96, I never managed to sort this out sorry. Things became very complicated using Celery with channels. I ended up backing away from it. – Marco Polo May 17 '20 at 09:19

Sending message from Celery task to Channels

1 Answers1