0

i have a 6gb httpd log file and i want to remove lines beginging in 66.249 (ip block of googlebot) i did have a 

SetEnvIf Remote_Addr "66\.249\.\." dontlog

entry in my httpd.conf file but it didnt seem to work

so is there a linux command like 

grep -removelines-starting "66.49"  acessslog
Cyrus
  • 84,225
  • 14
  • 89
  • 153
DOMDocumentVideoSource
  • 626
  • 1
  • 7
  • 16
  • Hi, If any of the following answer helped you then please acknowledge by accepting the answer by ticking the right sign beside the answer. You can also upvote others who are providing multiple approaches to solve the same problem. – P.... Jul 05 '17 at 05:53
  • To avoid future logging of those lines in the first place change the regex to `SetEnvIf Remote_Addr "66\.249\.[0-9]+\.[0-9]+" dontlog` and make sure the `CustomLog` looks somewhat like this: `CustomLog /var/log/apache2/access_log combined env=!dontlog` (add **env=!dontlog** to the end) – Holger Böhnke Aug 08 '18 at 15:57

2 Answers2

2

Using sed: Use -i flag if you make changes in file directly. 

sed  '/^66\.49/d' logfile

Using grep: This will print lines apart from lines starting with 66.49 

grep -v '^66\.49' logfile

Using awk:This will print lines apart from lines starting with 66.49 

awk '!/^66\.49/' logfile
P....
  • 17,421
  • 2
  • 32
  • 52
1

I can imagine sed is a better fit for this task. 

sed -i '/66\.249/d' ./acessslog

"d" is for deleting matched pattern, while -i is for overwriting input file.

metamorphling
  • 369
  • 3
  • 9
  • 1
    this will ALSO remove lines starting with `661249` or `663249` ....and so on. Reson: `.` represent any character in regex. – P.... Jun 30 '17 at 06:12
  • 1
    @PS. You are right, edited my answer! Have to be careful. – metamorphling Jun 30 '17 at 06:15
  • You should ancor the IP to the start of the line to avoid killing lines that accidently contain the pattern somewhere in the middle. Use pattern: `'^66\.249/d'`. Unless of course the IP is not at the start of line. – Holger Böhnke Aug 08 '18 at 16:06