5

I have a node cluster where the master responds to http requests. The server also listens to websocket connections (via socket.io). A client connects to the server via the said websocket. Now the client choses between various games (with each node process handles a game).

The questions I have are the following:

  • Should I open a new connection for each node process? How to tell the client that he should connect to the exact node process X? (Because the server might handle incoming connection-requests on its on)
  • Is it possible to pass a socket to a node process, so that there is no need for opening a new connection?
  • What are the drawbacks if I just use one connection (in the master process) and pass the user messages to the respective node processes and the process messages back to the user? (I feel that it costs a lot of CPU to copy rather big objects when sending messages between the processes)
InsOp
  • 2,425
  • 3
  • 27
  • 42

1 Answers1

6

Is it possible to pass a socket to a node process, so that there is no need for opening a new connection?

You can send a plain TCP socket to another node process as described in the node.js doc here. The basic idea is this:

const child = require('child_process').fork('child.js');
child.send('socket', socket);

Then, in child.js, you would have this:

process.on('message', (m, socket) => {
  if (m === 'socket') {
    // you have a socket here
  }
});

The 'socket' message identifier can be any message name you choose - it is not special. node.js has code that when you use child.send() and the data you are sending is recognized as a socket, it uses platform-specific interprocess communication to share that socket with the other process.

But, I believe this only works for plain sockets that do not yet have any local state established yet other than the TCP state. I have not tried it with an established webSocket connection myself, but I assume it does not work for that because once a webSocket has higher level state associated with it beyond just the TCP socket (such as encryption keys), there's a problem because the OS will not automatically transfer that state to the new process.

Should I open a new connection for each node process? How to tell the client that he should connect to the exact node process X? (Because the server might handle incoming connection-requests on its on)

This is probably the simplest means of getting a socket.io connection to the new process. If you make sure that your new process is listening on a unique port number and that it supports CORS, then you can just take the socket.io connection you already have between the master process and the client and send a message to the client on it that tells the client where to reconnect to (what port number). The client can then contain code to listen for that message and make a connection to that new destination.

What are the drawbacks if I just use one connection (in the master process) and pass the user messages to the respective node processes and the process messages back to the user? (I feel that it costs a lot of CPU to copy rather big objects when sending messages between the processes)

The drawbacks are as you surmise. Your master process just has to spend CPU energy being the middle man forwarding packets both ways. Whether this extra work is significant to you depends entirely upon the context and has to be determined by measurement.


Here's ome more info I discovered. It appears that if an incoming socket.io connection that arrives on the master is immediately shipped off to a cluster child before the connection establishes its initial socket.io state, then this concept could work for socket.io connections too.

Here's an article on sending a connection to another server with implementation code. This appears to be done immediately at connection time so it should work for an incoming socket.io connection that is destined for a specific cluster. The idea here is that there's sticky assignment to a specific cluster process and all incoming connections of any kind that reach the master are immediately transferred over to the cluster child before they establish any state.

jfriend00
  • 683,504
  • 96
  • 985
  • 979
  • [The documentation](https://nodejs.org/api/child_process.html#child_process_child_send_message_sendhandle_options_callback) points out that a whole server can be sent via `child_process.send`. I'm not sure whether or not we can send a WebSocketServer to another cluster. This functionality might help avoid reconnect the entire connections from crashes. – Lewis Mar 24 '16 at 17:45
  • @Tresdin - You aren't really sending a "whole server". You're sending the socket that is listening for incoming connections such that the other process can then listen for those incoming connections. In a typical installation, there usually is not a separate WebSocketServer because incoming webSocket connections usually share the same port (and thus server) as the regular web server HTTP connections. And, remember that all socket.io connections start life as an HTTP connection which then go through an upgrade process to negotiate changing the protocol to the webSocket protocol. – jfriend00 Mar 24 '16 at 17:59
  • @Tresdin - But, yes you could send a listening server socket to another process. – jfriend00 Mar 24 '16 at 18:00
  • Thanks for the clarification. I used to think that there is no way to rescue ws connections from being closed by a crashed process. Amazing news, indeed! – Lewis Mar 24 '16 at 18:17
  • @Tresdin - I rather doubt you're going to be able move an already established webSocket connection (e.g. fully initialized) to a new server because there is state associated with that connection above the TCP level that will not be moved. You can probably move an incoming HTTP connection that is a request to make a webSocket connection because that socket is still stateless in the HTTP mode. But, I don't think you can move it after it is established and converted from HTTP to webSocket protocol. It would be an interesting exercise to try though. – jfriend00 Mar 24 '16 at 18:20
  • @Tresdin - It's probably possible to write your own code that moves webSocket state from one server to another to go with the socket, but that would involve getting pretty intimate with the server-side webSocket implementation to know how to capture all of its state and then send that to the other server and reunite that with the socket you sent. – jfriend00 Mar 24 '16 at 18:25
  • 1
    "Possible" is good enough for me. If established connections can not be sent, I'm thinking about filing an issue then asking for an official method from [ws](https://github.com/websockets/ws) team. There must be a solution for this problem. – Lewis Mar 24 '16 at 18:53
  • @jfriend00 thank you for your answer. I decided to go with option 2. With this https://codesachin.wordpress.com/2015/07/20/how-to-deploy-multiple-node-js-socket-io-servers-with-nginx-and-ssl/ I did not had to open more ports than necessary. – InsOp Mar 25 '16 at 01:39