0

We have experienced some instability on our server lately. Checking log/messages there is a lot of messages like this

Apr 25 20:27:17 lillevinkel2 drupal: http://skobloggen.lillevinkelsko.no|1335378437|page not found|122.232.5.21|http://skobloggen.lillevinkelsko.no/http://www.magicptp.com/promote.php?id=dongxh|http://pulsreklamy.pl/|0||promote.php

They are from a lot of different ip adresses. I added in .htaccess that everything with http in should be denied. pasting the url from the log results in access denied in browser, but the log is still filling with this. What can I do to prevent this kind of spidering or stop the requests before they result in a page not found?

Tom
  • 219
  • 1
  • 4

1 Answers1

0

If it's a matter of the logs filling up, you can stop logging those entries using SetEnvIf. Something like:

SetEnvIf Request_URI "http" NoLog
CustomLog logs/access_log common env=!NoLog

More documentation here:

http://httpd.apache.org/docs/2.2/mod/mod_setenvif.html and http://httpd.apache.org/docs/2.2/env.html

cjc
  • 24,916
  • 3
  • 51
  • 70
  • That will clean up the log and is very useful. Thx. But I would also like to stop it with "access denied" before there is a "page not found". My access denied rewrite rule works when I paste the URL into the browser, but still it is coming new "page not found" messages. – Tom Apr 26 '12 at 20:00
  • Look at: http://httpd.apache.org/docs/2.2/howto/access.html#env You can do a similar `Deny from env=NoLog` with the same `SetEnvIf` pattern match. – cjc Apr 26 '12 at 20:04