We consume an XML Datafeed by sending cURL requests to the provider's server. They do not allow us to cache, so each pageview is a new cURL request. Twice this year, our server has crashed with the explaination from our server company being that the IP that belongs to the provider's domain is sending us a lot of traffic. Now, it seems to me that by "sending a lot of traffic" that might mean just thier answers to our requests, but then again, if it's actually them sending requests to us, I need to be sure of that before I start hassling their techs to fix a problem that might be on our end. If the problem is on our end, can I throttle the requests? (This is custom coded in PHP. If this is too much in one question, let me know and I'll split it into smaller chunks.)
EDIT: Here's what my hosting provider said: (They previously thought the /dev/sda drive was failing.)
Upon rebooting the server into single-user mode, the drives are not reporting any errors. Additionally, all of the filesystems are intact. We rebooted the server and everything came back online properly. I did see that there was a massive load spike on the server at the time that it stopped responding, and that around that time there were a lot of connections form the IP 208.215.218.15. There are also a lot of connections from that IP right now. This is the info on that IP:
Quick Technologies Inc. UU-208-215-218 (NET-208-215-218-0-1) 208.215.218.0 - 208.215.218.255 MCI Communications Services, Inc. d/b/a Verizon Business UUNET1996B (NET-208-192-0-0-1) 208.192.0.0 - 208.255.255.255
Connections from this IP quickly filled your network and I was forced to null-route connections from this IP in order to keep the network up. If this was somehow legitimate, let me know and I can unblock this IP.
At this time, your sites are up. If you have any questions, or there is anything else I can do for you, please let me know.
Then they gave me a log like this: (local ip changed for privacy)
root@fbs [~/system-snapshot/system-snapshot-HG-backup-20120604-144747]# grep 208.215.218.15 112.log tcp 0 0 123.123.123.198:45948 208.215.218.15:443 TIME_WAIT - tcp 0 0 123.123.123.198:45946 208.215.218.15:443 TIME_WAIT - tcp 0 0 123.123.123.198:45922 208.215.218.15:443 TIME_WAIT - tcp 0 0 123.123.123.198:45921 208.215.218.15:443 TIME_WAIT - tcp 0 0 123.123.123.198:45974 208.215.218.15:443 TIME_WAIT - tcp 0 0 123.123.123.198:45973 208.215.218.15:443 TIME_WAIT - tcp 0 0 123.123.123.198:46037 208.215.218.15:443 TIME_WAIT - tcp 0 0 123.123.123.198:46036 208.215.218.15:443 TIME_WAIT - tcp 0 0 123.123.123.198:46033 208.215.218.15:443 TIME_WAIT - tcp 0 0 123.123.123.198:46032 208.215.218.15:443 TIME_WAIT - tcp 0 0 123.123.123.198:46030 208.215.218.15:443 TIME_WAIT - tcp 0 0 123.123.123.198:46029 208.215.218.15:443 TIME_WAIT - tcp 0 0 123.123.123.198:46028 208.215.218.15:443 TIME_WAIT - tcp 0 0 123.123.123.198:46026 208.215.218.15:443 TIME_WAIT - tcp 0 0 123.123.123.198:46067 208.215.218.15:443 TIME_WAIT - tcp 0 0 123.123.123.198:46063 208.215.218.15:443 TIME_WAIT - tcp 0 0 123.123.123.198:46062 208.215.218.15:443 TIME_WAIT - tcp 0 0 123.123.123.198:46061 208.215.218.15:443 TIME_WAIT - tcp 0 0 123.123.123.198:46060 208.215.218.15:443 TIME_WAIT - tcp 0 0 123.123.123.198:46059 208.215.218.15:443 TIME_WAIT - tcp 0 0 123.123.123.198:40448 208.215.218.15:80 TIME_WAIT - tcp 0 0 123.123.123.198:40449 208.215.218.15:80 TIME_WAIT - tcp 0 0 123.123.123.198:40450 208.215.218.15:80 TIME_WAIT - tcp 0 0 123.123.123.198:40451 208.215.218.15:80 TIME_WAIT - tcp 0 0 123.123.123.198:40452 208.215.218.15:80 TIME_WAIT -
There were many, many similar lines below this. They finally answered in another reply:
Here's some random samples of network traffic for that IP, and the arrows show the direction (and amount of data) FROM your server or TO your server from that IP:
fbs.foobar.net => 208.215.218.15 5.84Kb 3.51Kb 3.19Kb <= 16.7Kb 181Kb 92.1Kb
fbs.foobar.net => 208.215.218.15 11.3Kb 23.1Kb 16.2Kb <= 0b 27.9Kb 152Kb ...
Currently there are 27 connections from 208.215.218.15, but the traffic is minimal at present, as seen above (5-16.7 Kbs)
So it looks like traffic from them to us is probably limited to responses to our curl requests, but I'm still not 100% clear on that, or why or if this caused the server to stop responding.
