Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have a long running batch process that outputs some debug and process information to stdout. If I just run from a terminal I can keep track of 'where it is' but then the data gets too much and scrolls off the screen.

If I redirect to output to a file '> out.txt' I get the whole output eventually but it is buffered so I can no longer see what it is doing right now.

Is there a way to redirect the output but make it not buffer its writes?

share|improve this question
1  
Could you please have a look at my (and @cnst 's) "debate" below, I'm guessing the only thing you want is to see the output in the same time as logging it to a file. If you found a solution, let us know about it ;) ! –  Benj Dec 2 '13 at 16:22

8 Answers 8

You can explicitly set the buffering options of the standard streams using a setvbuf call in C (see this link), but if you're trying to modify the behaviour of an existing program try stdbuf (part of coreutils starting with version 7.5 apparently).

This buffers stdout up to a line:

stdbuf -oL command > output

This disables stdout buffering altogether:

stdbuf -o0 command > output
share|improve this answer
    
aaarghhh... My Ubuntu has only 7.4 coreutils... :( –  Calmarius Feb 18 '13 at 11:38
    
@Calmarius: compiling coreutils should be quite easy. Just grab a new version from ftp.gnu.org/gnu/coreutils and give it a go. It's standard ./configure && make fare. You don't even have to install it afterwards, you can just use the stdbuf binary off src/. –  Eduardo Ivanec Feb 18 '13 at 14:52
    
double argh. i need this on an old centos distro AND OSX as well as Ubuntu. –  edk750 Apr 16 at 0:00

You may achieve line buffered output to a file by using the script command like so:

script -q /dev/null batch_process | tee output.log     # Mac OS X, FreeBSD
script -c "batch_process" /dev/null | tee output.log   # Linux
share|improve this answer

Personally I prefer piping output of a command I want to examine through tee.

script records too much information, including timing of key presses, and a lot of non-printable characters. What tee saves is much more human readable for me.

share|improve this answer
    
I would also add " | less" to the command line. –  HUB Jul 26 '11 at 15:43
3  
I'm pretty sure that tee is affected by buffering also. I frequently still get partial lines displayed by tee when splitting output from find commands. –  Magellan Jul 26 '11 at 16:19

Redirect the output into a file and follow the file with the tail -f command.

Edit

If this still suffers from buffering, then use the syslog facility (which is generally unbuffered). If the batch process runs as a shell script, you can use the logger command to do this. If the batch job runs in a scripting language, there should be a logging facility anyway.

share|improve this answer
1  
This is what I do right now but the process buffers the write to the file and that is just what I want to avoid. –  James Dean Jul 27 '11 at 14:54
    
See my edit for an updated proposal. –  wolfgangsz Jul 27 '11 at 15:12
    
This doesn't work for obvious reasons. Who the hell gives out This answer is useful for answers that don't work, and can't possibly work? –  cnst Nov 24 '13 at 9:14

try the script command; if your system has it, it takes a file name as argument, all text dumped to stdout gets copied to the file. It's very useful when a setup program requires interaction.

share|improve this answer
    
I know of the 'script -a out.txt' trick. I was wondering if there is any other way to make the writing process not buffer. –  James Dean Jul 27 '11 at 14:56

On Ubuntu, the unbuffer program (from the expect-dev) package did the trick for me. Just run:

unbuffer your_command

and it won't buffer it.

share|improve this answer

You can use the tee command, just magic !

someCommand | tee logFile.log will both display in the console and write into the log file.

share|improve this answer
    
Doesn't work. tee will not stop any buffering from taking place. –  cnst Nov 24 '13 at 9:15
1  
@cnst Effectively, tee will not avoid some buffering but will only allow you to have a look on what is the output. This is what @JamesDean wanted (as I undestood his question), but I think buffering is not really the problem here. If you have more details, let me know. –  Benj Nov 30 '13 at 10:14
    
He wanted to see output, and he's not getting any output (in an unbuffered manner), and yet you suggest to use tee to get a better view at the output that one isn't getting? –  cnst Nov 30 '13 at 17:56
1  
How I red @JamesDean question : "I redirect to output to a file '> out.txt'" means for me no console output, wait for process to complete while the entire output is redirected. When you are using > you don't see anything on the console. I guess @JamesDean is using the word "buffering" to describe that. I'll post a comment on his question for him to say more about what he wants. –  Benj Dec 2 '13 at 16:20

The easiest solution that I found (didn't need any third-party packages installed) was mentioned in a similar thread over on at Unix & Linux site: use the script command. It's old, and likely already installed.

$ script -q /dev/null long_running_command | print_progress       # FreeBSD, Mac OS X
$ script -q -c "long_running_command" /dev/null | print_progress  # Linux

Note that the first filename parameter for the script command is the log file to be written. If you simply run script -q your_command, you'll overwrite the command you indented to run with the log file. Check man script, to be safe, before trying it.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.