Parsing help in Perl

Question

My (existing) perl files creates a Log file using Log4Perl in the following format

[2011-11-21 08:50:22,406] States_Sync INFO Logger.pm:33 script starts
[2011-11-21 08:50:22,610] States_Sync ERROR Logger.pm:36 Error occurred ....
[2011-11-21 08:50:22,406] States_Sync INFO Logger.pm:33 ...
[2011-11-21 08:50:22,610] States_Sync ERROR Logger.pm:36 Error occurred ....
[2011-11-21 08:50:22,406] States_Sync INFO Logger.pm:33 ...
[2011-11-21 08:50:22,610] States_Sync ERROR Logger.pm:36 Error occurred ....

The above is only an example of my log file. I use the following formatter

$layout = Log::Log4perl::Layout::PatternLayout->new("[%d{ISO8601}] %c %p %F{1}:%L %m%n");

Currently I have to send an email in case of error.

Instead of modifying the existing script, I thought of parsing the generated log files only for Error and send all the messages related to "Error" from the log file as email

Is there any easy way of parsing the log file ?

Regards,

Karthik

any easy way of searching for some pattern like `ERROR` and collecting all from the same log file? — KK99, Dec 12 '11 at 13:51

score 3 · Accepted Answer · answered Dec 12 '11 at 14:02

3

use grep(1):

grep ERROR log.file

or use perl:

perl -ne 'print if /ERROR/' log.file

answered Dec 12 '11 at 14:02

tadmc

3,714
16
14

1

+1 just because I was thinking of putting this in a comment. ;) – TLP Dec 12 '11 at 14:55

score 1 · Answer 2 · answered Dec 12 '11 at 14:01

I might try this:

if ( /^[^:]+?\s+ERROR\s/ ) {
    # pull fields
    # send email
}

Try that one out and see if it gives you too many lines. More elaborate version might be:

if ( /^\[\d{4}-\d\d-\d\d \d\d:\d\d:\d\d,\d{3}\]\s+([\w\s])?\s+ERROR\s+/ ) {

but only if the data was more complicated.

score 0 · Answer 3 · answered Dec 12 '11 at 14:11

0

Since you're using log4perl, you could use the built-in methods there to check for an ERROR (or above) and send an e-mail based on that.

if ($layout->is_error()){
  # Put e-mail logic here
}

And if you wanted just error information and not warn, info, debug or trace you can do the following:

if($layout->is_error() && ! $layout->is_warn()){
  # Put e-mail logic here
}

answered Dec 12 '11 at 14:11

Dylan Northrup

161
1
9

unfortunately for this i have to change the existing code I am looking for like a wrapper – KK99 Dec 12 '11 at 14:37
If you can't modify the existing code, then using grep or perl regular expressions as stated above is the correct way to go. – Dylan Northrup Dec 12 '11 at 15:02

score 0 · Answer 4 · answered Dec 15 '11 at 08:31

You did not specify how often do you have to check the log file and send the e-mails. If it once a day then the solution provided by tadmc would be your best bet.

However, if you would like to automate it and send an e-mail after a specific interval of time (see maxinterval) for each new ERROR entry encountered, you may want to check the following.

NOTE #1 Adjust the interval and maxinterval as per your requirements to not flood people with e-mails NOTE #2 Run it in accordance to your log rotate intervals

#!/usr/bin/perl

use strict;
use warnings;
use File::Tail;

my @logs_to_email;
my $log_file = "file.log";

my $error_pattern = qr(^\[.*?\]\s*States_Sync\s*ERROR);

my $tail = File::Tail->new(
        name => $log_file,
        maxinterval => 60,
        interval => 10,
        adjustafter => 10,
);

while (defined (my $line = $tail->read)) {
        chomp $line;
        next if $line =~ /^\s*$/;
        next unless $line =~ $error_pattern;
        push @logs_to_email, $line;
        ##
        ##  put e-mail logic to send
        ##    @logs_to_email here
        ##
}

Parsing help in Perl

4 Answers4