-1

a log generates errors. it generates two very similar lines for 1 error I want to grep these errors but only one of the lines

for example:

Line is : Mar 21 15:33:04 VMP05 SMC_User: FATAL ECSDPROD 5210/SUPPORT/ECSD 21/03/17 15:33:04 VMD25 DIR_CHECK 0 FATAL File /data1/gmq6/in/29920991077061 is more than 10 minutes old

Line is : Mar 21 15:33:04 VMP05 SMC_User: FATAL ECSDPROD 5210/SUPPORT/ECSD 21/03/17 15:33:04 VMD18 DIR_CHECK 0 FATAL File /data1/sftp/out/26515991064454 is more than 10 minutes old

Mar 21 15:33:04 VMP05 SMC_User: FATAL ECSDPROD 5210/SUPPORT/ECSD 21/03/17 15:33:04 VMD25 DIR_CHECK 0 FATAL File /data1/gmq6/in/29920991077061 is more than 10 minutes old

Mar 21 15:33:04 VMP05 SMC_User: FATAL ECSDPROD 5210/SUPPORT/ECSD 21/03/17 15:33:04 VMD18 DIR_CHECK 0 FATAL File /data1/sftp/out/26515991064454 is more than 10 minutes old

but I only want to grep the lines without 'Line is'. I am using Hewlett Packard Linux

EDIT: this grep is needed within a tail -f :

    #!/usr/bin/ksh
echo "checking for last 10 fatals"
grep "FATAL ECSDPROD" /data1/log/startstop/MonitorDaemon.log|tail > /tmp/AH/linesDP.txt
grep "FATAL ECSD" /tmp/AH/linesDP.txt | grep -v "Line is"

echo "\n\n----------\n"
echo "checking for new fatals"
tail -f /data1/log/startstop/MonitorDaemon.log | grep "FATAL ECSD" | grep -v "Line is"
echo "about to exit"
exit 0

with the above the tail isn't updating, the script gets all the way down to the echo "checking for new fatals" then it won't tail the log

Community
  • 1
  • 1
Danny Piggott
  • 1
  • 1
  • try `tail...| stdbuf -o 0 awk '/FATAL ECSD/ && !/Line is/'` ... got idea from https://stackoverflow.com/questions/21617192/buffer-records-in-gnu-awk – Sundeep Mar 21 '17 at 16:26
  • Hi Sundeep, I get the following error ./fatalschkdanp.ksh[8]: stdbuf: not found As mentioned we are using Hewlett Packard Linux which isn't exactly the same as normal linux so might not have the stdbuf – Danny Piggott Mar 21 '17 at 16:31
  • stdbuf not on your machine then... as answered by @Terminator_NL, https://stackoverflow.com/questions/13858912/piping-tail-output-though-grep-twice should help – Sundeep Mar 21 '17 at 16:36
  • following comes back sorry grep: illegal option -- - – Danny Piggott Mar 21 '17 at 17:00

2 Answers2

0

Maybe this helps.

tail -f /data1/log/startstop/MonitorDaemon.log | grep --line-buffered "FATAL ECSD" | grep -v "Line is"

Elaboration: You can use the pipe symbol ( | ) multiple times in one command!

Source

Community
  • 1
  • 1
Terminator_NL
  • 193
  • 1
  • 1
  • 10
  • Hi, this doesn't work. Technically it does, but what I forgot to explain is I want it to be in a tail. and with this it won't update. full line tail -f /data1/log/startstop/MonitorDaemon.log | grep "FATAL ECSD" | grep -v "Line is" – Danny Piggott Mar 21 '17 at 16:03
  • 1
    Updated answer. The reason it didt work was because of a buffer built into grep, see: [source](http://stackoverflow.com/questions/13858912/piping-tail-output-though-grep-twice) – Terminator_NL Mar 21 '17 at 16:33
  • the following error comes back grep: illegal option -- - – Danny Piggott Mar 21 '17 at 17:00
  • Can you provide a log of the console output and what you put in? – Terminator_NL Mar 21 '17 at 17:33
  • #!/usr/bin/ksh echo "Checking For The Last 5 Fatals" grep "FATAL ECSDPROD" /data1/log/startstop/MonitorDaemon.log|tail > /tmp/AH/linesDP.txt grep "FATAL ECSD" /tmp/AH/linesDP.txt | grep -v "Line is" echo "\n\n----------\n" echo "Checking For New Fatals" tail -f /data1/log/startstop/MonitorDaemon.log | grep --line-buffered "FATAL ECSD" | grep -v "Line is" echo "about to exit" exit 0 – Danny Piggott Mar 22 '17 at 09:47
  • Checking For The Last 5 Fatals 5FATALS LIKE BELOW , 2MANY CHARACTERS 4COMMENT Mar 22 09:13:05 VMP05 SMC_User: FATAL ECSDPROD 5210/SUPPORT/ECSD 22/03/17 09:13:04 VMD17 DIR_CHECK 0 FATAL File /data1/gswd/out/26523991712579 is more than 10 minutes old ---------- Checking For New Fatals grep: illegal option -- - usage: grep [-E|-F] [-c|-l|-q] [-bhinsvwx] -e pattern_list... [-f pattern_file...] [file...] usage: grep [-E|-F] [-c|-l|-q] [-bhinsvwx] [-e pattern_list...] -f pattern_file... [file...] usage: grep [-E|-F] [-c|-l|-q] [-bhinsvwx] pattern [file...] about to exit – Danny Piggott Mar 22 '17 at 09:50
0

I think I might of solved this by using a combination of awk and grep. I am testing it at the moment but so far it works. 

    #!/usr/bin/ksh
echo "Checking For The Last 5 Fatals"
grep "FATAL ECSDPROD" /data1/log/startstop/MonitorDaemon.log|tail > /tmp/AH/linesDP.txt
grep "FATAL ECSD" /tmp/AH/linesDP.txt | grep -v "Line is"

echo "\n\n----------\n"
echo "Checking For New Fatals"
tail -f /data1/log/startstop/MonitorDaemon.log | awk '/FATAL ECS/' | grep -v "Line is"
echo "about to exit"
exit 0
Danny Piggott
  • 1
  • 1