Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have a file where numbers are continuously appended:

1
2
3
4

I want to calculate their mean, also continuously, i.e.:

1
1.5
2
2,5

I don't want to check file periodically, I want to it in the manner tail -f work - as soon as a line is appended, I perform mean calculations.

Is it possible?

UPD Question moved to http://stackoverflow.com/questions/9400306/countinuous-processing-of-tail-f-output

share|improve this question

closed as off topic by Chopper3, mdpc, Khaled, Zoredache, Ward Feb 22 '12 at 17:50

Questions on Server Fault are expected to relate to professional server, networking, or related infrastructure administration within the scope defined by the community. Consider editing the question or leaving comments for improvement if you believe the question can be reworded to fit within the scope. Read more about reopening questions here.If this question can be reworded to fit the rules in the help center, please edit the question.

    
Wait, how did you get a mean of 5? –  Jeff Ferland Feb 22 '12 at 17:06
    
tip: have a look at the tailf man :) –  Shadok Feb 22 '12 at 17:11
    
Is it possible YES! –  mdpc Feb 22 '12 at 17:13
    
>>Wait, how did you get a mean of 5; true. Screwed up –  Stas Feb 22 '12 at 17:33
1  
I am not aware of any tools to do this for you, but I know this should be trivial to do with a perl script. –  Zoredache Feb 22 '12 at 17:53

2 Answers 2

up vote 1 down vote accepted

Yes. Pipe tail -f to a script that handles your averaging. The pipe will never close and the script can process instantly for each line it receives... it will block until a line appears.

Also, one should keep in mind that it is possible to calculate a running average without having to add all the values each time. I've seen it that enough I feel the need to mention it.

#generator.pl
$| = 1; #immediate flush
while (1) {
    print int rand(100), "\n";
    sleep 1;
}

#average.pl
$| = 1; #immediate output flush
my $average = 0;
my $count = 0;
while (<>) {
    $average = ($average * $count + $_) / ($count + 1);
    $count++;
    print $average, "\n";
}

$ perl generator.pl > source &
[2] 15564
(reverse-i-search)`': ^C
$ tail -f source | perl average.pl
54
28
27.6666666666667
35
41

And, just for grins:

$tail -f source | awk '{total+=$0; count+=1; print total/count}'

That also has instant feedback. It seems to me that your issue is buffering by the application that is writing to the file that tail is reading from.

See http://www.pixelbeat.org/programming/stdio_buffering/ for info on that.

share|improve this answer
    
I have tried to do it with awk, but it didn't work, i.e. tail -f file.txt | awk {total+=$0; count+=1} END {print total/count} - and it just doesn't give any output –  Stas Feb 22 '12 at 17:31
    
@Stas ... as it shouldn't because you put that in the END block. Since the file never ends, it never prints. awk {total+=$0; count+=1; print total/count} –  Jeff Ferland Feb 22 '12 at 17:38
    
Yep, tried, but still hangs... –  Stas Feb 22 '12 at 17:44
    
The source is actually apache's access log. Probably I can't influence it's flushing... But I mean, if I can see it on the screen line by line, it must be a way to process it accordingly. Thanks for perl script. I will save them for exceptional case –  Stas Feb 22 '12 at 18:02
    
ok, I have figured out what was wrong. I used another awk after tail -f, which was buffered. Should be like tail -f source.txt | awk '{print $0; fflush()}' | awk '{total....}'. But thanks for your help - it did help a lot! –  Stas Feb 22 '12 at 20:26

Here's a simple solution using dc.

tail -f | dc -e '5k 0 d sc st [? lc 1 + sc lt + st lt lc / p c lax] sa lax'

I'm not sure whether dc supports tail recursion, if not this program will leak memory. dc has really bad documentation. :)

What, you don't like brainfuck? :)

Here's an easy solution in Ruby. tail -f | ruby -e 'sum=total=0.0; while line=gets; total += line.to_f; sum += 1; puts total/sum; end'

share|improve this answer
2  
looks like Brainfuck –  Stas Feb 22 '12 at 17:50

Not the answer you're looking for? Browse other questions tagged or ask your own question.