2

I have been trying to figure out what seems to be socket exhaustion on a production web server (Windows Server 2012). I have a powershell script, which is using Get-NetTCPConnection, to record the tcp ports that are in use.

The script is outputting that our processes are showing a large amount of sockets are with a remote address of 0.0.0.0:0 with a state of "Bound". There is no code that binds to this address either. The socket spike mostly begins at midnight, peaks at 4am, and returns to normal by 6am. The average number of sockets with remote address of 0.0.0.0:0 at the peak is 1500, which is way too high.

There are no scheduled processes either that are running at this time, and we have very little load.

Am I looking in the wrong place, or maybe I need to use a different tool? Maybe its not even a socket exhaustion problem in the first place? I don't know.

The c# exception message I am dealing with that lead me to believe it is this:

An operation on a socket could not be performed because the system lacked sufficient buffer space or because a queue was full

Joshua5822
  • 341
  • 2
  • 10

1 Answers1

2

The 0.0.0.0 IP address is reserved to specify all IP addresses on the local machine. Even though you're probably reading them as the foreign address, that is only the name of the other end of the socket connection - that can still be the local IP address.

And now, why 0.0.0.0 and not 127.0.0.1? The point is that the local machine can have multiple IP addresses assigned. When a listening socket is established, it can specify that it is listening for incoming connections on any IP address on this machine. Therefore, the connections you are seeing are the ones established with such listening sockets on your computer.

For diagnostic purposes, I was using the CurrPorts utility which could tell me different things about socket connections, including the process which is holding the connection. You can find it at address https://www.nirsoft.net/utils/cports.html

Zoran Horvat
  • 10,924
  • 3
  • 31
  • 43
  • The tool has been more useful in logging then what I was getting from the powershell script. It doesn't show me the Bound state nonsense and I instead am seeing a large amount of Time_Wait, which is more what I expected. Still investigating it though as I need to tweak the frequency of logging better. – Joshua5822 Feb 28 '18 at 17:24
  • 1
    @Joshua5822 Consider upvoting. I am amazed I've been using Get-NetTcpConnection for years instead of CurrPorts. Zoran, you da man. – John Zabroski Sep 27 '19 at 14:41