3

I'm a bit of noob when it comes to deploying web apps and wanted to make sure a little app I'm building will work with the tech I'm trying to use.

I have some experience with flask, but have only ever used the test server. My understanding is that with nginx or apache, if I write a flask app, each user who visits my website could get a different instance of the flask app, exactly how that will work is a little confusing to me.

The app I want to make is similar to chatrooms/a game like "among us". When a user comes to the website, they join a big "lobby" and can either join a "room" that already exists, or launch a new room and generate a code/ID that they can pass to their friends so that their friends can join the same session (I think a socketio "room" can be used for this).

However, if each client is connected to their own flask instance, will every server instance be able to see the "rooms" on the other instances? Suppose my app becomes really popular and I want to scale the lobby across multiple machines/AWS instances in the future, is there anything I can do now to ensure this works? Or is scaling across multiple machines equivalent to scaling across instances on a single machine as far as the flask-socketio/nginx stack is concerned.

Basically, how do I ensure that the lobby part of the code is scalable. Is there anything I need to do to ensure every user has the ability to connect to rooms with other users even if they get a different instance of the flask app?

CasualScience
  • 661
  • 1
  • 8
  • 19

1 Answers1

5

I will answer this question specifically with regards to the Socket.IO service. Other features of your application or third-party services that you use may need their own support for horizontal scaling.

With Flask-SocketIO scaling from one to two or more instances requires an additional piece, a message queue, which typically is either Redis or RabbitMQ, although there are a few more options.

As you clearly stated in your question, when the whole server is in a single instance, data such as which room(s) each connected client is in are readily available in the memory of the single process hosting the application.

When you scale to two or more instances, your clients are going to be partitioned and randomly assigned to one of your servers. So you will likely end up having the participants that are in a room also spread across multiple servers.

To make things work, the server instances all connect to the message queue and use messages to coordinate complex actions such as broadcasts to a room.

So in short, to scale from one to more instances, all you need to do is deploy a message queue, and change the Flask-SocketIO server to indicate the location of the queue. For example, here is the single instance server instantiation:

from flask_socketio import SocketIO

socketio = SocketIO(app)

And here is the initialization with a Redis message queue running on localhost's default 6379 port:

from flask_socketio import SocketIO

socketio = SocketIO(app, message_queue='redis://')

The application code does not need to be changed, Flask-SocketIO does all the coordination between instances for you by posting message on the queue.

Note that it does not really matter if the instances are hosted in the same server or in different ones. All that matters is that they connect to the same message queue so that they can communicate.

Miguel Grinberg
  • 65,299
  • 14
  • 133
  • 152
  • Thank you very much for the clear answer. I'll dig a bit more into the flask-socketio MQ docs. – CasualScience Feb 01 '21 at 19:44
  • @Miguel Thanks for your informative reply, but I'm puzzled by one part of your answer: "With Flask-SocketIO scaling from one to two or more instances requires an additional piece, a message queue"...So my question is: is this the only way to scale flask-socketio, or is this the most recommended way? I'm a long time user of uwsgi (thanks!), and am hitting issues of scale with my setup, and evaluating the best ways forward. For example, is using nginx for proxy forwarding to multiple instances no longer recommended? – stephenrs Apr 21 '21 at 06:27
  • @stephenrs the Socket.IO server is stateful. The message queue allows the different instances to share their state with their peers. It is required, as I mentioned in my reply. – Miguel Grinberg Apr 21 '21 at 22:13
  • @Miguel Thanks, but I indeed understand the stateful nature of socketio, and maybe this is a bit OT, but sharing state is not a requirement for my case at this phase. So, more specifically, is running flask-socketio on multiple uwsgi workers a supported configuration without a MQ, in cases where sharing state (and intra-instance coordination) is not a concern? Or, does having multiple flask-socketio workers somehow break things in all cases? – stephenrs Apr 22 '21 at 06:58
  • @stephenrs feel free to test any deployment strategy that you'd like to use, but I don't consider dropping the message queue a supported configuration under any circumstance. You may also want to read the documentation regarding the sticky session requirements (which uWSGI's load balancing does not support). – Miguel Grinberg Apr 22 '21 at 10:51
  • @Miguel Thanks again for the guidance, although I'll note that sticky sessions is also not a requirement for the particular use case I'm focused on. I've actually tested running socketio on multiple workers and I haven't yet noticed any issues, but I was wondering if certain conditions or fundamental incompatibilities might surface problems down the road that my testing might not easily reveal. Please correct me if I'm wrong, but it sounds like the answer is "yes". – stephenrs Apr 22 '21 at 15:48
  • @stephenrs There is nothing more than I can tell you. You are using something that is a derivative of Socket.IO, so basically you are on your own. If it works for you, I'm not going to stop you, but I'm also not going to go out of my way to fix bugs or address issues that occur in your set up. – Miguel Grinberg Apr 24 '21 at 10:35
  • "I don't consider dropping the message queue a supported configuration under any circumstance" <-- Is there a technical rationale for this statement, beyond the presumption that every project needs to coordinate activities between instances, thus requiring things like shared state and sticky sessions? Knowing the answer to this would be very helpful. I've heard/read the same thought in various places, but there is never a direct answer to the question of why. I suppose I should have just started there for a more direct path to a straight answer to my original question. – stephenrs Apr 25 '21 at 08:27
  • @Miguel Asked another way...would a more accurate way to phrase your original comment be "With Flask-SocketIO scaling from one to two or more instances requires an additional piece, a message queue *IF* you need to coordinate activities between instances"? – stephenrs Apr 25 '21 at 08:28
  • @stephenrs As I said above, there is nothing more than I can tell on this subject. I have never used Socket.IO in the way that you describe, that would not be Socket.IO it would be a derivative. If you want to use something like that you are free to do so, but you will be completely on your own. I have no interest in discussing or supporting such a solution. The Socket.IO protocol is publicly available. Read it, and then decide if what you want to do can be done without the queue. – Miguel Grinberg Apr 25 '21 at 10:55