1

I need to pipe certain log entries into a perl script, but I can't get it to work using ARGV or STDIN. 

tail -f messages | grep --line-buffered "auth failure:" | awk '{print $1,$2,$3,$10}' | test3.pl

Perhaps something is being buffered but it appears nothing is making it to test3.pl, but if I leave off the | test3.pl then I see what should be going in to perl: 

Feb 3 16:09:36 [user=someusername]
Dan
  • 5,081
  • 1
  • 18
  • 28
Xi Vix
  • 1,381
  • 6
  • 24
  • 43

3 Answers3

5

awk will buffer output by default if it's not connected to a terminal. Call fflush() or system("") in your awk script.

mob
  • 117,087
  • 18
  • 149
  • 283
  • thanks ... you were right ... that was the problem ... awk was buffering ... I had a feeling but didn't know about fflush(); – Xi Vix Feb 04 '12 at 02:57
3

Sounds like awk performs no buffering or line buffering when connected to a terminal, and performs block buffering when connected to something other than a terminal. This is pretty standard behaviour, and it's the reason you have to pass --line-buffered to grep.

You need to find a way to disable awk's buffering. I don't know how to do that, but I can provide a Perl alternative.

tail -f messages \
   | perl -lne'BEGIN{$|=1} /auth failure:/ && print join " ", (split)[0,1,2,9]' \
      | test3.pl

See also: File::Tail.

ikegami
  • 367,544
  • 15
  • 269
  • 518
  • thanks ... that worked ... I wish I knew how to program command line perl like that ... I haven't found a good reference – Xi Vix Feb 04 '12 at 02:50
  • @xivix, `-n` and `-l` are documented in [perlrun](http://perldoc.perl.org/perlrun.html). – ikegami Feb 04 '12 at 06:02
2

The input from the pipe will be in your STDIN. What makes you think it isn't there? You can read it using

while (<>) {
  print;
}

Also, you can do all of those steps in one simple Perl program.

Borodin
  • 126,100
  • 9
  • 70
  • 144
  • I get nothing ... even using your code suggestion above ... so how would I use a perl script to monitor a real-time log? – Xi Vix Feb 03 '12 at 23:38
  • Try it with something simpler, like `cat *file* | test3.pl`. And do you have a #! line in your perl program? Otherwise you will have to run `cat *file* | path/to/perl test3.pl` – Borodin Feb 03 '12 at 23:54
  • that worked ... so it must have something to do with awk even though awk provides output until I add the pipe to perl. – Xi Vix Feb 04 '12 at 02:52
  • ok I ran with your idea and now have a working program. However, it keeps constantly giving itself a new process ID when I do a pgrep on it. It's as if it is executing itself over and over. If so, the variables I have set up in the program are not going to work. Here it is: – Xi Vix Feb 06 '12 at 21:31
  • ` exec("tcpdump -A -n -nn -i eth0 not host 72.215.138.71 and not host 66.210.32.132 and not host 66.210.221.11 and not host 66.210.221.12 and not host 66.210.221.13 and not host 66.210.221.14 and not host 66.210.221.15 and not host 66.210.221.16 | grep --before-context=4 authentication > /scripts/ad.tmp"); open Tail, '/usr/bin/tail -f /var/log/messages | grep --line-buffered "auth failure:" | awk "{print \$3,\$10; fflush();}" |' or die "Pipe failed: $!\n+"; while (){ ... do stuff ... } exit(0); ` – Xi Vix Feb 06 '12 at 21:34
  • sorry for the formatting ... I can't figure out how to add line breaks in these comments – Xi Vix Feb 06 '12 at 21:40