3

My C# program is receiving a lot of data via multiple sockets and it appears that I have problems that the TCP cache on the Windows Kernel is getting overloaded.
This can be seen by the following results from the tcpdump from the server:

18:02:18.963370 IP xxxx > yyyy.zzz: . ack 10017011 win 0

Which data structure/strategy would you use to cache the read data to avoid the overload on C#?
I am currently working with a delegate for every socket in a seperate thread which is called after every read record, but this seems to be a problem, so that I might need to split it into two threads.
Or should I look at tweaking the TCP parameters on the Windows side? I am using Windows 7.
Are there any books for the kernel optimization?
I was really blindsided that the program got slower even though the load is around 20-25 % for the full machine and 50-60 % on the highest loaded core.

laalto
  • 150,114
  • 66
  • 286
  • 303
weismat
  • 7,195
  • 3
  • 43
  • 58
  • 1
    Very similar to http://stackoverflow.com/questions/5234277/c-1-socket-client-continous-data/5234427. I would recommend the same thing in your case. – Jim Mischel Mar 08 '11 at 20:46
  • Yes - indeed. I am still amazed that besides tcpdump on the counterparty there are no real indicators of an overloaded TCP stack. It is apparently the bottleneck even if the application itself looks healthy when looking at the performance monitors in Windows. – weismat Mar 09 '11 at 05:09

1 Answers1

5

Without knowing too much about the architecture of your program, I would guess that your program itself is the bottleneck. If your application doesn't use a background worker to collect TCP messages and store them to be consumed by the code that does the "heavy lifting", then your program is reading the existing buffer, then processing what it has while the TCP buffer refills. This vicious cycle can only have one ending; eventually the buffer will overflow while waiting for your program to finish processing what it got last time.

I would look at refactoring your TCP reception into an ETL-type model. You need one thread whose job is to listen for TCP input, extract the data from the buffer and enqueue it in a collection that's visible to the rest of the program. It should do nothing else other than yank data out of the buffer and shove the raw messages into this queue, and it should do this as fast as it can.

With the received messages in memory, the rest of your program can take almost as long as it wants to consume each message. You still have to ensure you don't overload this in-memory queue, but as a single .NET object can reference up to 2GB worth of memory, you're much less likely to run into trouble. If you ARE overflowing this in-program queue, you need to look at either making the processing more efficient, or throwing more threads at this second stage of the process.

KeithS
  • 70,210
  • 21
  • 112
  • 164
  • The program is normally not crashing, but the server is occassionaly dropping a connection. The current architecture has one thread per socket and a delegate for every item - I guess I will need to test several caching ways, but a blocking collection sounds like a viable candidate. Depends a bit if the parsing/uncompression needs to be one the reader thread or not. Thanks for the advice. – weismat Mar 09 '11 at 04:47
  • After some tests I have ended up working with a shared object with byte array, offset and used length. I tried sharing a memory stream collection, but the overhead looked too big (after a few out of memory exceptions). – weismat Mar 09 '11 at 10:47