-3

I have an application in C# that is a TCP server listening to a port. GPS devices connect to this port. The application is accepting the TCP client and creating a new thread for each client. The client ID in maintained in a hash table that is updated when a client is connected. this was all working fine until around 400 units. Once the number of units increased, the server was unable to handle all connections. The connections are being continuously dropped and once in awhile leads eating up the server CPU and memory and brings it down. Work around was to open another instance of the TCP server listening to a different port and diverted some units to that port. Currently some 1800 units are somehow running in 8 different ports. The server is extremely unstable and units are still unable to stay connected. Facing too many issues on a daily basis. Also using remoting to send settings via the remoting port - this is working only sometimes.

Please help by giving a solution for TCP socket/threading/thread pooling etc. that is both scalable and robust and can in a single port.

This TCP server is running in Windows server 2008 R2 Enterprise with IIS7 and SQL server 2008.
Processor: Intel Xenon CPU E3-1270 V2 @3.50GHz
RAM: 32GB System: 64-bit operating system

Thanks Jonathan

Jonathan
  • 23
  • 5
  • One-thread-per-client simply will not scale on Windows. You need to use one of the several asynchronous API models for networking in .NET. Note that no matter what, you will always have a limit. But with the basic async I/O (`BeginXXX()` methods or the `XXXAsync()` methods on `NetworkStream` class) you should easily be able to handle up to tens of thousands of connections, and if you use the more efficient `XXXAsync()` methods on the `Socket` class (which is different than async stream methods), you should be able to handle 100's of thousands. – Peter Duniho Feb 08 '15 at 18:13
  • Note also that the above assumes the rest of your own code is written well. There are lots of ways you can mess up scalability in a networking server. You haven't provided enough information for anyone to know what your _actual_ problem is, but we can say for sure that your current implementation has at least _one_ known scalability problem. – Peter Duniho Feb 08 '15 at 18:15
  • @PeterDuniho can this socket connection scenario apply to desktop Windows OS as well, e.g. Windows 7 Pro x64. When running a socket server on desktop OS, are thousands of connections still reasonable? – khargoosh Aug 16 '16 at 00:22

2 Answers2

2

Basically, don't use a thread per socket; use one of the async APIs (BeginReceive / ReceiveAsync), or some kind of socket polling (Socket.Select for example, although note that this is implemented in a very awkward way; when I use this, I actually use P/Invoke to get to the raw underlying API). Right at this moment, I have > 30k sockets per process talking to our web-sockets server (which is implemented via via Socket). Note that for OS reasons we do split that over a few different ports - mainly due to limitations of our load-balancer:

enter image description here

Marc Gravell
  • 1,026,079
  • 266
  • 2,566
  • 2,900
  • @Mark I am sure your environment is much larger than what I am working on.. Is it possible for you to come down to my level and help me out? How do I send my code? I need help in various places in my code. – Jonathan Feb 17 '15 at 15:07
  • @Jonathan beyond the scale of "thread per socket doesn't work" (which already applies to you), scale is largely irrelevant - the problem is simply that you need to switch to async IO – Marc Gravell Feb 18 '15 at 08:56
  • @Mark Gravell - I have switched to asynch IO... am revamping my TCP server....... – Jonathan Feb 19 '15 at 06:12
  • @MarcGravell Can someone reasonably expect this sort of performance when running a socket server on a desktop Windows OS? – khargoosh Aug 16 '16 at 00:24
  • @khargoosh one of the configuration differences between the desktop OS and the server OS is the amount of handles / ephemeral ports / etc made available by default for things like networking, because (oddly enough) the server OS is expected to be *used as a server*. You can possibly reconfigure the desktop network stack to get similar levels, but if you're writing a server, you should probably use a server OS – Marc Gravell Aug 16 '16 at 09:36
  • @MarcGravell what about a side case: a peer to peer network, where over time each client has a comparable number of tcp connections being accepted and attempted outbound. In your experience would a thousand socket connections be unreasonable? – khargoosh Aug 16 '16 at 10:18
  • @khargoosh I don't know; I only use desktop OS for development; I'm a server person :) Your P2P question is valid, but I simply don't know the answer. – Marc Gravell Aug 16 '16 at 10:23
0

One thread per connection is not a really good idea specially when you have to handle 100s of client concurrently.Asynchronous is the way to go with some buffer pooling/managing. If you are looking for something to start with asynchronous sockets have a look at this basic implementation if you are looking for something complete Take a look at this(explanation: Here) If you are willing check this out too.

In C# you can go with classical BeginXXX/EndXXX methods. Microsoft also have a High Performance Socket API which can be leveraged using XXXAsync methods. A few articles which explain the High Performance Socket API Here and Here

Abdullah Saleem
  • 3,701
  • 1
  • 19
  • 30
  • Thank you people for giving me the right direction. Thanks Abdullah for the specific links to codes. Working on implementing that on my end... will let you know how it goes... – Jonathan Feb 09 '15 at 06:03
  • Hello All, I have got the server and client application running in my test environment. There's a lot more to do until I can take it into realtime. – Jonathan Feb 10 '15 at 08:02
  • I do not have control over how my actual client functions. It establishes a connection to the server, then it sends data anytime within the next 2 minutes. This code is based on how the included client program behaves. On debugging with another simple client where the client is simply connecting(no data sent yet) it is giving an Out of Memory exception at receiveSendToken.theDataHolder.dataMessageReceived = new Byte[receiveSendToken.lengthOfCurrentIncomingMessage]; and dataMessageReceived is shown as null. What can I do? – Jonathan Feb 10 '15 at 19:00
  • I have connected my unit as a client to the server. I am stuck at line 117 in the SocketListener class. bool incomingTcpMessageIsReady = messageHandler.HandleMessage(receiveSendEventArgs, receiveSendToken, remainingBytesToProcess); is supposed to send a true, only then further processing will start. I found that inside handle Message, receiveSendToken.lengthOfCurrentIncomingMessage is showing a very large value like 1677777 where as if the tailored client is connected it has the value of 5 only. so it keeps on receiving the data looks like it is waiting for 1677777 bytes. Please help! – Jonathan Feb 11 '15 at 09:37
  • @Jonathan you are framing the data...right? If you don't feel like posting your code here....you can contact me using the email address in my profile – Abdullah Saleem Feb 11 '15 at 16:32
  • could u down load the code at http://www.codeproject.com/Articles/83102/C-SocketAsyncEventArgs-High-Performance-Socket-Cod ? I am not using their client but another trial client that is sending the words Hello There while it connects. Am unable to understand how the PrefixHandler will work and then how the MessageHandler will work. The number of bytes sent are obviously 11. – Jonathan Feb 12 '15 at 10:35
  • @Jonathan as tcp is stream based so there is no way to distinguish between different packets so in order to over come this issue message framing is used. The length of the message to be sent is prefixed before the message so that the receiving end knows exactly how much bytes to receive. Explanation: http://blog.stephencleary.com/2009/04/message-framing.html – Abdullah Saleem Feb 12 '15 at 15:48
  • the point is that I do not have control over the client. It is a third party client and it sends various types of messages including keep alive and only on of the messages has the length as a prefix, that too not the first four bytes. – Jonathan Feb 12 '15 at 16:13
  • @Jonathan then you will have to write a custom message handler according to the clients protocol – Abdullah Saleem Feb 12 '15 at 18:03