1

I maintain a Django Channels v1 app with six Consumers multiplexed over one WebSocket connection. I'm upgrading to Django Channels v2, where connection (de)multiplexing is no longer supported; see GitHub Issue #825 - (Re)Implement Multiplexing. If I want to multiplex, I can use channels-demultiplexer, channelsmultiplexer, or write my own. I'm leaning towards re-implementing multiplexing because I think it would entail changing less of our existing code.

Before doing that, I'm doing some due diligence to find out whether multiplexing is even an important optimization.

Given a Redis-backed Django 2 + Channels 2 app in a load balanced environment behind Nginx with Daphne (currently planning mostly SyncConsumers) and Gunicorn, I think that increasing the number websocket connections by 3x-5x would only negligibly increase Redis RAM, app server RAM, and Daphne CPU, and also negligibly increase browser RAM and CPU, but that's just an educated guess.

  1. How should I think about the cost of each additional WebSocket connection a.) in Django Channels v2 and b.) in the browser?
  2. I have not seen any web apps that were designed to use many WebSocket connections on a single page. Have you? If so, is it performant? How many connections did it utilize?
  3. What would you do?

Stack Overflow posts that address adjacent but different questions:

  1. How to use multiple websocket connections using Django Channels?
  2. Django Channels 2 implementing Multiplexer/Demultiplexer like Channels 1
  3. Reusing an existing websocket in Django Channels
Myer
  • 3,670
  • 2
  • 39
  • 51

1 Answers1

3

As the author of channelsmultiplexer I hope i have some useful insights.

There is one key thing to remember with Channelv2 each consumer instance has 1 run-roop so while it is handling a message is will not be able to handle another one (they get queued up). In particular if you have actions on your consumer that take time (db, or other) the consumer will not get other messages while it is doing work. When using channelsmultiplexer the blocking does not affect other multiplexed consumer instances. If there are changes/additions you need to channelsmultiplexer feel free to create an issue on the repo i am happy to make changes as needed to fit more peoples use cases.

How should I think about the cost of each additional websocket connection a.) in Django Channels v2 and b.) in the browser?

Yes since unless your tunnelling ws over HTTP2 (not something channels support but nginx could help you year if you are sure all your clients are http2) the users browser will likely limit the number of concurrent HTTP connections you can have open. Many browsers will limit you to about 7 open connections but this could change at any time, and mobile browser might be more strict (they tend to be)

Matthaus Woolard
  • 2,310
  • 11
  • 13
  • If I have multiple daphne instances behind a load balancer, will Channels v2 really make it so each user is capped at one synchronous operation? If so, can you point me to the place in the code or docs where it does that? – Myer Apr 18 '21 at 18:45
  • 1
    not each connection but each consumer. (so unless you re using a multiplexer that does mean each connection yes) https://github.com/django/channels/blob/main/channels/consumer.py#L58 this `await_many_dispatch` https://github.com/django/channels/blob/f2fc01be0f735205d9803abe7fbda131028e0746/channels/utils.py#L32 on L40 you can see it used the same loop for all the callable. – Matthaus Woolard Apr 18 '21 at 19:28
  • 1
    its not really a synchronous operation, they are still async (do not block other consumer instance) but the block other messages for this consumer. The reasons for this is that you can now store state on the consumer instance (it is a regular class) so having multiple things running at once reading/writing that state is not a good idea. – Matthaus Woolard Apr 18 '21 at 19:30
  • Thanks for all this info; this is very helpful, though I find it challenging to use the right terminology with Channels v2. 1. `run-roop` is Scooby-Doo for `run-loop`, right ;) ? 2. You mention that there's one run-loop for each _consumer instance_ - this covers one _particular connection_, e.g. one user, right? 3. `channelsmultiplexer` uses a subclass of `AsyncJsonWebsocketConsumer`. My consumers need to use the Django ORM, and the Channels v2 docs suggest I should `SyncConsumer` for this case. Can I still use `channelsmultiplexer` with `SyncConsumer`s? – Myer Apr 19 '21 at 14:28
  • 1) yes sorry (should write these things on my phone) 2) yer you can, `SyncConsumer` is basicly and Async consumer were all the lifetime methods have been wrapped to be sync. The multiplexer will be `Async` but the upstream consumers can be any JSON web-socket consumer. – Matthaus Woolard Apr 19 '21 at 19:03