1

I have a problem with one of my servers. Google opens lots of http connections to the apache server and basically performs a slowloris attack.

This netstat call results in the following output

netstat -plant|grep :80|awk '{print $5}'|cut -d: -f1|sort|uniq -c|sort -n

 11 209.85.227.93
 14 209.85.227.190
 30 209.85.229.118
494 209.85.229.141

All of these ip's belong to the google network. I've already forbidden Google to access any of the content that is saved on the server by using robots.txt but it has no effect on the insane amount of connections. What should I do?

Tom O'Connor
  • 27,480
  • 10
  • 73
  • 148
Philip
  • 165
  • 3
  • 13
  • 2
    I'd stick the IP addresses into an iptables rule to block them - not putting this as an answer because my iptables skills aren't up to scratch - but an iptables drop will stop them using resources at your end – EightBitTony Jul 03 '11 at 19:39
  • 1
    Great Idea, Thank you! BTW the command is: `/sbin/iptables -I INPUT -s 209.85.229.141 -j DROP` – Philip Jul 03 '11 at 19:46
  • google has how to on this [link](https://support.google.com/webmasters/answer/156449?hl=en). – Sharad Chhetri Aug 25 '13 at 20:23

1 Answers1

8

Go to google webmaster tools, put in throttling.

Several other spiders respect the Crawl-delay directive in robots.txt, but Googlebot doesn't.

Community
  • 1
TomTom
  • 51,649
  • 7
  • 54
  • 136