Get value from next N rows of a file

Question

I'm having problems intercepting the contents of the lines above what I'm reading $lines[0] as following foreach loop

my $IN_DIR  = "/tmp/appo/log";              # Input Directories
my $jumprow = '<number of row to skip>';    # This is a value

foreach my $INPUT ( glob( "$IN_DIR/logrotate_*.log" ) ) {

    open( my $fh, '<', $INPUT ) or die $!;

    while ( <$fh> ) {

        next unless $. > $jumprow;

        my @lines = split /\n/;
        my $i     = 0;

        foreach my $lines ( @lines ) {

            if ( $lines[$i] =~ m/\A#\d.\d.+#\d{4}\s\d{2}\s\d{2}\s\d{2}:\d{2}:\d{2}:\d{3}#\+\d+#\w+#\/\w+\/\w+\/Authentication/ ) {

                # Shows only LOGIN/LOGOUT access type and exclude GUEST users

                if ( $lines[ $i + 2 ] =~ m/Login/ || $lines[ $i + 2 ] =~ m/Logout/ && $lines[ $i + 3 ] !~ m/Guest/ ) {

                    my ( $y, $m, $d, $time ) = $lines[$i] =~ /\A#\d.\d.+#(\d{4})\s(\d{2})\s(\d{2})\s(\d{2}:\d{2}:\d{2}:\d{3})/;

                    my ( $action ) = $lines[ $i + 2 ] =~ /\A(\w+)/;
                    my ( $user )   = $lines[ $i + 3 ] =~ /\w+:\s(.+)/;

                    print "$y/$m/$d;$time;$action;$user\n";
                }
            }
            else {
                next;    # Is this next technically necessary according to you?
            }

            $i++;
        }
    }

    close( $fh );
}

The Tie::File module could help me

my $IN_DIR  = "/tmp/appo/log";              # Input Directories
my $jumprow = '<number of row to skip>';    # This is a value

foreach my $INPUT ( glob( "$IN_DIR/logrotate_*.log" ) ) {

    tie @lines, 'Tie::File', $INPUT, mode => O_RDONLY;
            or die $!;

    my $i = $.;

    next unless $i > $jumprow;

    foreach my $lines ( @lines ) {

        if ( $lines[$i] =~ m/\A#\d.\d.+#\d{4}\s\d{2}\s\d{2}\s\d{2}:\d{2}:\d{2}:\d{3}#\+\d+#\w+#\/\w+\/\w+\/Authentication/ ) {

            # Shows only LOGIN/LOGOUT access type and exclude GUEST users

            if ( $lines[ $i + 2 ] =~ m/Login/ || $lines[ $i + 2 ] =~ m/Logout/ && $lines[ $i + 3 ] !~ m/Guest/ ) {

                my ( $y, $m, $d, $time ) = $lines[$i] =~ /\A#\d.\d.+#(\d{4})\s(\d{2})\s(\d{2})\s(\d{2}:\d{2}:\d{2}:\d{3})/;

                my ( $action ) = $lines[ $i + 2 ] =~ /\A(\w+)/;
                my ( $user )   = $lines[ $i + 3 ] =~ /\w+:\s(.+)/;

                print "$y/$m/$d;$time;$action;$user\n";
            }
        }
        else {
            next;    # Is this next technically necessary according to you?
        }

        $i++;
    }
}

Could you tell me if my declaration with Tie::File is correct or not? This is only a part of my master script as indicated in following guide mcve

Actually without tie, my master scripts works only with $lines[0], it doesn't take value from $lines[$i+2] or $lines[$i+3]

Can you include sample input and expected output as well please? — simbabque, Aug 20 '18 at 13:58
@simbabque Hi :) Input file is it just the same of this https://stackoverflow.com/q/51636352/1195443 — clarkseth, Aug 20 '18 at 14:01
Has Dave actually solved your problem in that last question you just linked? If so, you might want to accept his answer, or tell us how it didn't help. We do pay attention to these things around here, and some of us feel unappreciated if you accept their free help but don't give them credit. ;) — simbabque, Aug 20 '18 at 14:04
Please show us _actual_ literal output. I have no idea what's in those variables. — simbabque, Aug 20 '18 at 14:04
@simbabque Dave did not solve the problem, I tried to explain myself but in the end the question took a bad turn, to get a short version of the script. In these two weeks I studied a bit of things, I improved the script and I think I can meet the requirements using this form (never used before). — clarkseth, Aug 20 '18 at 14:09
@simbabque I only noticed now that if I use the tie function, the value of $. does not report the line read at that time. Probably the mistake is just that. (My literal output is empity right now, with tie function) — clarkseth, Aug 20 '18 at 14:19

Borodin · Accepted Answer · 2018-08-20T18:46:01.673

It looks like you're getting very lost here. I've written a working program that processes the data you showed in your previous question; it should at least form a stable basis for you to continue your work. I think it's fairly straightforward, but ask if there's anything that's not obvious in the Perl documentation

use strict;
use warnings 'all';
use feature 'say';
use autodie;  # Handle IO failures automatically

use constant IN_DIR => '/tmp/appo/log';

chdir IN_DIR; # Change to input directory
              # Status handled by autodie

for my $file ( glob 'logrotate_*.log' ) {

    say $file;
    say '-' x length $file;
    say "";

    open my $fh, '<', $file; # Status handled by autodie

    local $/ = "";           # Enable block mode

    while ( <$fh> ) {

        my @lines = split /\n/;

        next unless $lines[0] =~ /
            ^
            \# \d.\d .+?
            \# (\d\d\d\d) \s (\d\d) \s (\d\d)
            \s
            ( \d\d : \d\d : \d\d : \d\d\d )
        /x;
        my ( $y, $m, $d, $time ) = ($1, $2, $3, $4);
        $time =~ s/.*\K:/./;  # Change decimal point to dot for seconds

        next unless $lines[2] =~ /^(Log(?:in|out))/;
        my $action = $1;

        next unless $lines[3] =~ /^User:\s+(.*\S)/ and $1 ne 'Guest';
        my $user = $1;

        print "$y/$m/$d;$time;$action;$user\n";
    }

    say "";
}

output

logrotate_0.0.log
-----------------

2018/05/24;11:05:04.011;Login;USER4
2018/05/24;11:04:59.410;Login;USER4
2018/05/24;11:05:07.100;Logout;USER3
2018/05/24;11:07:21.314;Login;USER2
2018/05/24;11:07:21.314;Login;USER2
2018/05/26;10:48:02.458;Logout;USER2
2018/05/28;10:00:25.000;Logout;USER0

logrotate_1.0.log
-----------------

2018/05/29;10:09:45.969;Login;USER4
2018/05/29;11:51:06.541;Login;USER1
2018/05/30;11:54:03.906;Login;USER4
2018/05/30;11:59:59.156;Logout;USER3
2018/05/30;08:32:11.348;Login;USER4
2018/05/30;11:09:54.978;Login;USER2
2018/06/01;08:11:30.008;Logout;USER2
2018/06/01;11:11:29.658;Logout;USER1
2018/06/02;12:05:00.465;Logout;USER9
2018/06/02;12:50:00.065;Login;USER9
2018/05/24;10:43:38.683;Login;USER1

Thanks your answer is really very informative and undoubtedly resolutive, I immediately run tests to integrate your logic to the script — clarkseth, Aug 20 '18 at 16:19
@clarkseth: There should be nothing to "integrate". This code should be a complete solution. It does everything from all versions of your code. — Borodin, Aug 20 '18 at 18:48
@clarkseth: It would be far better to record the date/time last read, and ignore blocks that are less than that when next reading the same file. It would amount to just another `next unless` in the `while` loop, together with code to store and retrieve the date/time from the most recent block. Perhaps you can even use the file you're already writing and read the last record from it? — Borodin, Aug 20 '18 at 21:13
I had skipped this solution because I thought it was computationally more complex. I do a test :) thanks — clarkseth, Aug 20 '18 at 21:23
Thank you for all <3 i've completed today my script with more functionalities — clarkseth, Aug 22 '18 at 16:15
@clarkseth: You may want to submit your code to [*Code Review*](https://codereview.stackexchange.com) as you're clearly not fluent with Perl yet and you may learn a lot from the feedback there. That applies especially if you're hesitant to reveal what you've written! — Borodin, Aug 22 '18 at 16:38

Get value from next N rows of a file

1 Answers1

output