Is it possible to create a single socket, single thread, TCP/IP packet reader for multiple clients?

Question

A typical server application would create a socket for each incoming connection spawn a new thread.

However is it possible to do the demuxing yourself in a single thread? What I want is that the application keeps a state machine for each end point which deserializes message as data becomes available in a single thread.

Threading is the simplest, and I have a working implementation of that. However each thread adds overhead fairly quickly, and lingering connections will require resources. What I want is that instead for each connection a state machine builds up a message (which the threaded code already does anyway) and upon completion dispatches the deserialized message to a worker queue.

Is this possible in .NET? P/Invoke would also be an acceptable solution.

Answer 1

A typical server application would create a socket for each incoming connection spawn a new thread.

That is not a true statement. Very few servers — only those which need not scale to large numbers of connected clients, and not even all of those — will dedicate an entire thread to a single connection.

I would say only the most rudimentary servers would use a thread-per-connection design. I suppose that since there are a lot more experimental servers (i.e. hobbyists learning to write network code) than production servers, the sheer numbers might be on the side of thread-per-connection.

But IMHO only production servers are really relevant to the question, as they show what good design and implementation would be. And for those, thread-per-connection is definitely going to be in the minority.

However is it possible to do the demuxing yourself in a single thread?

Using Socket.Select() one can have a single thread dedicated to the handling of several sockets.

However, more typical would be to use one of the several asynchronous programming APIs available for use with sockets, which uses I/O completion ports to farm out handling of I/O operations to a pool of threads dedicated for the purpose. This allows concurrency but efficient use of the threads to minimize context-switching.

My preferred approach with the current .NET 4.5 and C# 5.0 features is to wrap the socket in an instance of NetworkStream, so that I can use the ReadAsync() and WriteAsync() methods, which in turn allow the use of await in the C# code. This makes the asynchronous code a lot easier to read and implement.

But you can use e.g. Socket.BeginReceive() or Socket.ReceiveAsync() (the latter being useful for servers that require extremely high scalability…even the former is still much more scalable than thread-per-connection though) just as well.

Regardless of API used, the typical server implementation will include a state object class for each connection. This can be done with thread-per-connection design as easily as with the async I/O designs. Your state machine would reside in that state object class, so that the processing of I/O operations on the socket can use the state machine.

Answer 2

Use the Socket.Select method.

When you call the method, you pass it three lists of sockets. One is the list of sockets you're waiting for reads, one is a list you're waiting for writes (if the write buffer is full you will have to wait to write), and one is a list you're waiting for errors. You also tell it the maximum amount of time to wait, and it will block that long until one of the sockets is ready. When the function returns, it will modify the lists to remove the sockets that aren't ready.

When you create your read array, put the master/mother socket in so that you will know when there's a new connection. Also put in any existing connections. Put all of those in the error array also. For the write array, you only need to put sockets in when the write buffer is full. Probably this is something you can ignore in your first iteration, but it will be important when you start putting serious traffic through.

When Socket.Select() returns, you need only loop through the sockets remaining in the read list and process the data as your server requires. Loop through the writes and push out any remaining queued data. Loop through the errors and close those sockets or handle errors. Remember to put the idle sockets back in the list before you call select again!

Answer 3

Absolutely, the trick is to use asynchorous IO (i.e. BeginAccept, EndAccept, BeginRead, EndRead, BeginWrite, and EndWrite). This allows you to handle multiple clients without the need of threads. This is the way that react works.