-1

Our site is smashed by Google bot and Facebook:

66.249.90.50 - - [03/Apr/2017:09:26:41 +1000] "GET /news/?vmowv= HTTP/1.1" 200 48925 "-" "Mediapartners-Google"
66.249.90.49 - - [03/Apr/2017:09:26:42 +1000] "GET /news/?dmsdv= HTTP/1.1" 200 48920 "-" "Mediapartners-Google"
66.249.90.51 - - [03/Apr/2017:09:26:55 +1000] "GET /news/?gnqho= HTTP/1.1" 200 48921 "-" "Mediapartners-Google"

or

66.220.145.244 - - [03/Apr/2017:09:24:32 +1000] "GET /news/?fcyaz= HTTP/1.1" 200 48928 "http://l.facebook.com/l.php?u=http%3A%2F%2F9nl.it%2Fuxnp.dttwx%3Ffcyaz&h=ATMHNW9hkl6TONap_bRWuW2wMvkYVGe5ODTBEUAiP9AROqHHKZgwzgfRJ-FZkHIUh-QawYl_hFIH24AYXx8yocm74ZkOOPVnjX34jwJjLTl7YUnPhAlvWQ" "Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36"
66.220.145.244 - - [03/Apr/2017:09:24:31 +1000] "GET /news/?ptflu= HTTP/1.1" 200 48928 "http://l.facebook.com/l.php?u=http%3A%2F%2F9nl.it%2Fuxnp.fkngs%3Fptflu&h=ATOPijn1YdMSu_Vet64L1UbIYOfTCxcHcVs2Z5pjRCsCVTsfsqs1pK4tz6Wh0pZWkQEX6trHVBdpXCBThJubSEVcUJZC0B4al-zfu61bUUR4xg6ZoJh-gQ" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_1) AppleWebKit/536.25 (KHTML, like Gecko) Version/6.0 Safari/536.25 MenuTab"
66.220.145.244 - - [03/Apr/2017:09:24:32 +1000] "GET /news/?mxnis= HTTP/1.1" 200 48926 "http://l.facebook.com/l.php?u=http%3A%2F%2F9nl.it%2Fuxnp.fqfwd%3Fmxnis&h=ATNxQw6NkIgw4y65TZni-OzJq1SCpSmzdkdKU9c2KEBFe-9C5DfgbWtaJ_-vdU9Gnho-yC4ZnZXMGTKTLiSuBzjkFIBNeKY2lGnkUT_c5MsjWKqmcziSDA" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36"

I assume there is a page somewhere has all these wired links /news/?random_string. Facebook and Google try to crawl all these links. Is there anyway I can trace down the root cause?

starchx
  • 533
  • 10
  • 24

1 Answers1

0

It sounds like you may have mod_rewrite enabled and probably is answering some unexpected URLs. Check your .htaccess file to see what rules might be present.

user7749674
  • 148
  • 6