Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I'm actually trying to write a shell script that logs the output of a command to a file, but since the command takes a long time to complete (about 15 minutes), I would like to start parsing the output of the command (content of the file) before the command is completed, so I can send messages to the standard output (the user), like:

10% complete
45% complete

and so on.

Program steps

  1. Redirect the command to a file: $(command) > $FILE
  2. Start reading and parsing the output ($FILE) before the command is finished.

I thought of using parallel programming, but I havent't got the hang of it.

share|improve this question

4 Answers 4

Use tail and pipe that into your parser.

share|improve this answer

tail -f will follow the file as it being created.

try it with a log file.

tail -f /var/log/messages for example .

you'll see log file outputted to console as they appear in the log.

share|improve this answer

Pipes, Tee, and Tail (tee being at the heart of the question as it duplicated standard output. If sed were the file you were using to parse the data, an example would be:

kbrandt@k$ a=0; while [[ a -lt 10 ]]; do
> echo foo; a=$(( $a + 1));
> done | tee -a unprocessed | sed 's/foo/bar/' > processed
kbrandt@kbrandt$ cat unprocessed 
foo
foo
foo
...
kbrandt@kbrandt$ cat processed 
bar
bar
...

You can then run it as a job and tail processed, or just tail -f processed in another window.

share|improve this answer
up vote 1 down vote accepted

I've actually solved the problem using tail and the Stack Overflow question Ending tail -f started in a shell script.

# Solution
($(command) > ${FILE}) &
tail -f --pid=$! ${FILE} | while read line
do
      echo $line
      # Can parse each line output here
      # Example: send errors to log file
      if echo $line | grep -qi "error"
      then
            echo $line >> ${LOG}
      fi
done

Thank you all for the help.

share|improve this answer
    
You probably don't want the dollar sign at the beginning of the line you're backgrounding. –  Dennis Williamson Mar 18 '10 at 0:16
    
Edited to do exactly that (I had the dollar sign at the wrong place). Thank you for the spotting it. –  Isabelle Mar 22 '10 at 11:21

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.