How to catch whatever shell outputs?

Question

We're designing a shell in the browser that will stream everything that shell outputs. stdout, stderr, anything.

however, just as this little example would demonstrate to you,

git clone https://github.com/joyent/node.git >./git.out 
cat ./git.out

most of the output is not given to ./git.out. here is little video, if you want to see what's happening http://screencast.com/t/7zTdPfq7Pr7

We need commands like 'top','git' to work, any ideas how to get them ?

(outputting into a file is for illustration purposes only, we will stream every line of output to another system, we know for top it wouldn't make sense but I hope my point is clear enough for the ones who can provide insight. thanks.)

Answer 1

Most programs have two outputs: stdout and stderr. stdout is where the "main" information goes. stderr is used for, well, errors. It's also used for various non-important output, and also things like user prompts sometimes.

To capture all of it, you need to redirect stderr to stdout.

somecmd arg1 arg2 arg3 >somefile 2>&1

Or to capture stderr to a seperate file:

somecmd arg1 arg2 arg3 >stdout.txt 2>stderr.txt

Be aware that it IS possible for a program to use other outputs than stdout / stderr. It can determine your console and basically "force" output there. However this is extremely uncommon, and the above will work in 99% of situations.

One thing to keep in mind if you redirect both stdout and stderr to the same file is that stdout and stderr buffer differently, so it won't look the same in a log file as it will on the screen (lines will be out of order). Also, for your specific instance, your output is going to look like garbage. git does a lot of terminal manipulation (for the progress indicator) and in a log file it's going to look pretty awful.

EDIT

From man git-clone:

--progress
Progress status is reported on the standard error stream by default when it is attached to a terminal, unless -q is specified. This flag forces progress status even if the standard error stream is not directed to a terminal