3

I have a file where numbers are continuously appended:

1
2
3
4

I want to calculate their mean, also continuously, i.e.:

1
1.5
2
2,5

I don't want to check file periodically, I want to it in the manner tail -f work - as soon as a line is appended, I perform mean calculations.

Is it possible?

UPD Question moved to https://stackoverflow.com/questions/9400306/countinuous-processing-of-tail-f-output

Community
  • 1
Stas
  • 157
  • 1
  • 8

2 Answers2

2

Here's a simple solution using dc.

tail -f | dc -e '5k 0 d sc st [? lc 1 + sc lt + st lt lc / p c lax] sa lax'

I'm not sure whether dc supports tail recursion, if not this program will leak memory. dc has really bad documentation. :)

What, you don't like brainfuck? :)

Here's an easy solution in Ruby. tail -f | ruby -e 'sum=total=0.0; while line=gets; total += line.to_f; sum += 1; puts total/sum; end'

akramer
  • 556
  • 2
  • 6
1

Yes. Pipe tail -f to a script that handles your averaging. The pipe will never close and the script can process instantly for each line it receives... it will block until a line appears.

Also, one should keep in mind that it is possible to calculate a running average without having to add all the values each time. I've seen it that enough I feel the need to mention it.

#generator.pl
$| = 1; #immediate flush
while (1) {
    print int rand(100), "\n";
    sleep 1;
}

#average.pl
$| = 1; #immediate output flush
my $average = 0;
my $count = 0;
while (<>) {
    $average = ($average * $count + $_) / ($count + 1);
    $count++;
    print $average, "\n";
}

$ perl generator.pl > source &
[2] 15564
(reverse-i-search)`': ^C
$ tail -f source | perl average.pl
54
28
27.6666666666667
35
41

And, just for grins:

$tail -f source | awk '{total+=$0; count+=1; print total/count}'

That also has instant feedback. It seems to me that your issue is buffering by the application that is writing to the file that tail is reading from.

See http://www.pixelbeat.org/programming/stdio_buffering/ for info on that.

Jeff Ferland
  • 20,547
  • 2
  • 62
  • 85
  • I have tried to do it with awk, but it didn't work, i.e. tail -f file.txt | awk {total+=$0; count+=1} END {print total/count} - and it just doesn't give any output – Stas Feb 22 '12 at 17:31
  • @Stas ... as it shouldn't because you put that in the END block. Since the file never ends, it never prints. `awk {total+=$0; count+=1; print total/count}` – Jeff Ferland Feb 22 '12 at 17:38
  • Yep, tried, but still hangs... – Stas Feb 22 '12 at 17:44
  • The source is actually apache's access log. Probably I can't influence it's flushing... But I mean, if I can see it on the screen line by line, it must be a way to process it accordingly. Thanks for perl script. I will save them for exceptional case – Stas Feb 22 '12 at 18:02
  • ok, I have figured out what was wrong. I used another awk after tail -f, which was buffered. Should be like tail -f source.txt | awk '{print $0; fflush()}' | awk '{total....}'. But thanks for your help - it did help a lot! – Stas Feb 22 '12 at 20:26