Perl weird chomp and newline

Question

I'm having weird things when extracting line from file.

Lines are coming from SSH commands sent on a router and saved into a file. They are looking like : saved_commands

    FastEthernet0 is up, line protocol is up
    Helper address is not set
    FastEthernet1 is up, line protocol is up
    Helper address is not set
    FastEthernet2 is up, line protocol is down
    Helper address is not set

So, in my PERL script, when i'm reading this file, I make this :

while (<LIRE>) {
    $ifname = "";
    $etat = "";
    $myip = "";
    if (/line protocol/) {
        @status = split(/,/, $_) ;
        @interface = split(/ /, $_);
        $ifname = $interface[0];
        for ($index = 0; $index <= $#status; $index++) {
            @etat_par_if = split(/ /, $status[$index]);
            $etat .= pop(@etat_par_if);
        }
    }
    if (/Helper|helper/) {
        @helper = split(/ /, $_) ;
        $myip = pop(@helper);
        if ($myip =~ /126/) {

        }
        else {
            $myip = "not set";
        }
    }
    if ($myip eq ""){
        $myip = "not set";
    }       
    if ($ifname ne ""){
    print "$ifname ; $etat ; $myip \n";
    }
}
close(LIRE);

The output should be :

FastEthernet0 ; upup ; not set
FastEthernet1 ; upup ; not set
FastEthernet2 ; updown ; not set

But unfortunately, the output is more like :

FastEthernet0 ; upup
 ; not set
FastEthernet1 ; upup
 ; not set
FastEthernet2 ; updown
 ; not set

I guess there's a newline somewhere, probably at the end of each interface line. But i tried several things, like chomp($_), or even

$_ =~ s/
//;

But things get weirder. EDIT : I tried other answers, having same issue. Using Borodin's answer :

my ($etat, $ifname, $myip);

open(DATA,$fichier) || die ("Erreur d'ouverture de $fichier\n") ;
while (<DATA>) {

    if (/line protocol/) {
        $ifname = (split)[0];
        my @status    = split /,/;
        for (@status) {
          $etat .= (split)[-1];
        }
    }
    elsif (/helper/i) {
        my @helper = split;
        $myip = (split)[-1];
        $myip = 'not set' unless $myip =~ /\d/;
    }

    if ($ifname and $myip) {
      print "$ifname ; $etat ; $myip\n";
      $etat = $ifname = $myip = undef;
    }

}
close(DATA);

The output is like

FastEthernet0 ; upup ; not set
Vlan1 ; downdowndowndowndowndownupupupdownupdownupdownupdownupdownupdownupdownupup ; 126.0.24.130
Loopback0 ; upup ; not set
Loopback1 ; upup ; not set
Tunnel100 ; upupupupupup ; not set
Dialer1 ; upup ; not set
Tunnel101 ; upup ; not set
Tunnel200 ; upup ; not set
Tunnel201 ; upup ; not set
Tunnel300 ; upup ; not set

We are getting closer to it, what happened to the others FastEthernet interfaces ?

Added code change with chomp. It's like the interface disappeared or some weird things happens — Gui O, Jul 31 '14 at 09:20
try putting your `chomp($_)` at the start of the loop to remove the new line char before you start processing the line. — Chris Doyle, Jul 31 '14 at 09:21
Moved chomp($_); right after the while() { is giving the same output — Gui O, Jul 31 '14 at 09:23
`use Data::Dumper; $Data::Dumper::Useqq = 1; print Dumper($_); print Dumper($/);`. This will show you what is on the end of the line and what `chomp` will attempt to remove. — RobEarl, Jul 31 '14 at 10:18
Your file is from a Windows system and has CRLF as its line terminators. See my answer below. — Borodin, Jul 31 '14 at 10:35
You really shouldn't edit someone's answer into your question. If you need clarification, comment on that answer. — RobEarl, Jul 31 '14 at 10:51
It's the only answer that solved my first issue. As i'm trying to go forward with it, i used it in my question. i Would have let the old stuff, but it'll make a very big post. Do you want it ? — Gui O, Jul 31 '14 at 10:54
@GuiO: That would be because you have multiple *"line protocol"* lines that don't have a correpsonding *"helper"* line. The program prints the output only when it has found the information from *both* types of line. I've amended my answer, try it now. It just ignores all interfaces that aren't followed by a helper. If you need any more changes then we need to see a sample of the real data file. — Borodin, Jul 31 '14 at 11:01
Well, i need to see all interfaces, even those which don't have helpers, and for these ones, i'll set a string "no helper". I need it to put datas into DB Your answer is working perfectly for what you told me. Can you just rework it a little bit for what i said above please ? — Gui O, Jul 31 '14 at 11:03

Borodin · Accepted Answer · 2014-07-31T11:24:41.033

The problem is that you are processing a file from a Windows system that has CRLF at the end of each line instead of just LF (newline). chomp removes the newline, but the carriage-return remains and messes up your output. You can get around this by using s/\s+\z// instead of chomp.

But you can avoid the problem with line terminators altogether by using split with a single space string (not a regex). That is the default if you don't pass any parameters at all, and it splits on any combination of whitespace (which includes newlines and carriage-returns), ignoring any leading spaces.

This program seems to do what you want. It makes use of regular expressions instead of split to get straight to the correct part of the data without using temporary arrays.

use strict;
use warnings;

my $fichier = 'fichier.txt';

open my $lire, '<', $fichier or die "Erreur d'ouverture de $fichier: $!";

my ($ifname, $etat, $myip);

while (<$lire>) {

   if (/line protocol/) {

      if ($ifname) {
         printf "%s ; %s ; %s\n", $ifname, $etat, $myip // 'not set';
         $myip = undef;
      }

      ($ifname) = /(\S+)/;
      $etat = join '', /(\S+)\s*(?:,|\z)/g;
   }
   elsif (/helper.*\s([\d.]+)/i) {
      $myip = $1;
   }

}

if ($ifname) {
   printf "%s ; %s ; %s\n", $ifname, $etat, $myip // 'not set';
}

fichier.txt

FastEthernet0 is up, line protocol is up
Helper address is not set
FastEthernet1 is up, line protocol is up
Helper address is not set
FastEthernet2 is up, line protocol is down
Helper address is 126.0.24.130
FastEthernet3 is up, line protocol is down
FastEthernet4 is up, line protocol is down
Helper address is 126.0.24.128
FastEthernet5 is up, line protocol is down

output

FastEthernet0 ; upup ; not set
FastEthernet1 ; upup ; not set
FastEthernet2 ; updown ; 126.0.24.130
FastEthernet3 ; updown ; not set
FastEthernet4 ; updown ; 126.0.24.128
FastEthernet5 ; updown ; not set

That's it. Working as i wanted. Thanks a Borodin and Everyone. — Gui O, Jul 31 '14 at 11:13
I think I've realised what you really need. A *"line protocol"* record should always be displayed. If it has a corresponding *"helper"* record with a valid IP address then the third output field shows that address, otherwise it shows `not set`. — Borodin, Jul 31 '14 at 11:16

anurag · Answer 2 · 2014-07-31T09:29:11.607

I am not getting anything weird in your code but you are not using chomp anywhere in the code.

Just add chomp $_; at the very start of your while loop.

Try this Code:

while (<DATA>) {
    chomp;
    $ifname = "";
    $etat = "";
    $myip = "";
    if (/line protocol/) {
        @status = split(/,/, $_) ;
        @interface = split(/ /, $_);
        $ifname = $interface[0];
        for ($index = 0; $index <= $#status; $index++) {
            @etat_par_if = split(/ /, $status[$index]);
            #chomp @etat_par_if;
            $etat .= pop(@etat_par_if);
        }
    }
    if (/Helper|helper/) {
        @helper = split(/ /, $_) ;
        $myip = pop(@helper);
        if ($myip =~ /126/) {

        }
        else {
            $myip = "not set";
        }
    }
    if ($myip eq ""){
        $myip = "not set";
    }       
    if ($ifname ne ""){
    print "$ifname ; $etat ; $myip \n";
    }
}
close(DATA);


__DATA__
FastEthernet0 is up, line protocol is up
Helper address is not set
FastEthernet1 is up, line protocol is up
Helper address is not set
FastEthernet2 is up, line protocol is down
Helper address is not set

and this is the output:

FastEthernet0 ; upup ; not set 
FastEthernet1 ; upup ; not set 
FastEthernet2 ; updown ; not set

I did it in the previous comments. Nothing happens about the output. Having this weird interface name in between helper and etat — Gui O, Jul 31 '14 at 09:26
It's pretty awesome because i don't have the same output as you. Still have my buggy things... I can't write to you all my file describing my interfaces (internet address is... Broadcast address is... MTU is...) So you're telling my it's my input file the issue ? — Gui O, Jul 31 '14 at 09:36
Can you provide an exmaple of the whomplete line and maybe jsut change any real ip addresses to something else like 172.10.0.0. Its a lot easier for people to help if they know the full picture. — Chris Doyle, Jul 31 '14 at 09:44

score 0 · Answer 3 · answered Jul 31 '14 at 09:56

0

Try this $etat =~ s/[\n]+$//; or if that fails $etat =~ s/[\r\n]+$//;

I suspect you have one of these as the end of your newlines in the file being read in. I also suspect a single chomp may not work if the line ends in \r\n.

answered Jul 31 '14 at 09:56

Tim Wright

101
4

score -1 · Answer 4 · answered Jul 31 '14 at 09:51

-1

Without knowing the actual input file exact details the below is a suggestion and is written based on the contents of the input file you have provided in your post.

use strict;
use warnings;

open (my $file, '<', 'LIRE.txt') or die "unable to open file : $!\n";
while (<$file>) {
        chomp();
        if (/line protocol/){
                my ($ifname) = /^([A-Za-z0-9]+)\s/;
                my @status = map { /\s([A-Za-z]+)$/ } split(',');
                print "$ifname ; ", @status;
        } elsif (/[Hh]elper/){
                my $ip = /\s(\w+)$/;
                print " ; ", $ip =~ /126/ ? $ip : "not set", "\n";
        }
}
close($file);

answered Jul 31 '14 at 09:51

Chris Doyle

10,703
2
23
42

`/[Hh]elper/` is better written `/helper/i`. And you have no way of knowing what characters are in the OP's interface names and you can't just assume alphanumeric. Just `my ($ifname) = /(\S+)/` is fine. The first parameter of `split` is a regex and you should mark it as such, like so `split /,/`. – Borodin Jul 31 '14 at 10:34
You have no way of knowing whether the IP address ends at the end of the line. And it certainly doesn't consist of just `\w` characters. Furthermore `my $ip = /\s(\w+)$/` will set `$ip` to *true* or *false* and not the IP address itself. Something like `my ($ip) = /(\S+)\s*$/` is what you need. There is also no need for `chomp` if you write things this way. – Borodin Jul 31 '14 at 10:42

Perl weird chomp and newline

4 Answers4