0

I run a moderately active website on a dedicated server. In the course of protecting against (D)DOS attacks, I ran netstat to get a count of active connections. This produced a rather unusual result in that it showed 1000+ connections from the httpd port of the server's public ip-address to many different seemingly random ports on the same ip-address. That is, the output looks like this:

Proto   Recv-Q    Send-Q  Local Address         Foreign Address      State
tcp     0         0       [server's ip]:60284   [server's ip]:80     TIME_WAIT
tcp     0         0       [server's ip]:60267   [server's ip]:80     TIME_WAIT
... [1000+ just like this]

The state of these is TIME_WAIT but that may be because I run a firewall that doesn't allow access to these ports. This server does communicate with a database on another machine but that should all be done through a private ip. So if this is valid communication, I'd like to figure that out so I can set it to use the private ip (where the ports are accessible).

Does anyone know why the machine would attempt to communicate with itself over its public ip? How would I determine what process is initiating this? I tried to look at these ports via lsof but that did not return any information.

The only unusual thing that our server runs with httpd is the rather outdated persistent-perl program (to cache perl scripts in memory). However, I'm quite certain that would not involve use of the network and there is no information that indicates how that would work.

Thanks in advance for any advice!

cubiclemonk
  • 1
  • 1

1 Answers1

4

I think you'll find that those connections are to your HTTP port, not from it. The other port numbers (60284 etc) are the ephemeral port numbers automatically allocated to the client sides of the connections.

Whichever end of the connection (client or server) closes it is the one that enters the TIME_WAIT state after it is closed.

So, it looks like there's some process on your machine that is connecting to your webserver quite frequently, and closing the connection from the client side (I'd guess either closing before making an HTTP request, or making an HTTP 1.1 keepalive request then closing)

Some possible ways to track this down:

  • If you can catch one of them while the connection is in an ESTABLISHED state, netstat -antp (as root) will show you the process ID for the socket.

  • Have a look in your webserver access/error logs for requests coming from your server's IP. These may shed some light on where they are coming from.

  • Use tcpdump or wireshark to trace packets from your server's IP to itself on port 80, and have a look at what is actually being transmitted (you'll also be able to definitively see which way the connection is being opened).

andytech
  • 311
  • 1
  • 4
  • Thanks! It turns out that the requests seem to be coming from mod_pagespeed, as disabling that stopped the problem. I'll try to figure out what it's doing. – cubiclemonk Mar 02 '13 at 06:20