CentOS 5.x

I'm trying to build a shell script that searches data provided via stdin. Here's an example of the input stream:

Date: 1/1/11 
Time: 12:00 AM 
Foo: 12345 
Foo1: dskjflsdkjflksdjlfds 
Foo2: 123456789 
Foo3: kdsjflskdjflkjsdlkfjsdlkjflksdjflkjsdklfjlksdjflk

This information doesn't exist in a file, it would only be sent to the script in real-time as stdout from another app.

I want the sript to look at the data and parse out the values for Foo: and Foo2: storing those as variables for use later in the script.

My revised script attempt is this:

while read data; do
        SearchCriteria1=$(echo "$data" | grep "Foo: " | cut -c 5-)
        SearchCriteria2=$(echo "$data" | grep "Foo2: " | cut -c 6-)
        echo $SearchCriteria1 >> test.1
        echo $SearchCriteria2 >> test.2

The finished script won't actually use test.1 or test 2 files. I just have them listed here for easier examples.

In this example, I would expect test.1 to have:


In this example, I would expect test.2 to have:


When I test this though, both test.1 and test.2 are blank and I know the data has the valid info in it.

I'm missing something obvious. Can someone please clarify?

  • Please show us an example input and expected output? In your above example, don't you grep from the $data? – quanta Aug 13 '12 at 3:42
  • You're grepping for "Foo" followed by a space but your input file has "Foo" followed by a colon. – Alan Curry Aug 13 '12 at 5:05
  • @AlanCurry User error on my part when writing the serverfault question. I updated the question and they correctly match on my test system (sorry for the typos - I've been banging my head against this for awhile and am getting tired). – Mike B Aug 13 '12 at 5:08
  • 2
    now when I run what you have posted it works for me. I copied the first section to infile, the second section to scriptfile, ran chmod +x scriptfile;./scriptfile<infile and got the results. It also outputs a blank line for each non-matching input line, but we can fix that... [ -n "$SearchCriteria1" ] && echo "$SearchCriteria1" >> test.1 – Alan Curry Aug 13 '12 at 5:11
  • @AlanCurry Strange the example is working for me too. There must be a typo in the original. I'll look again. – Mike B Aug 13 '12 at 5:21

Think about where your greps are getting their input from. You've given them no input filename, so they're reading from standard input. What is their standard input? There is no redirection for it, and there is no pipe going into the greps, so they are running with the inherited standard input of the loop.

The read is executing only once. It reads the first line into $data, which you never use anywhere (that should be a clue that something is wrong). Then the first grep runs, it consumes all of the input, and the second grep is attached to the same input so it gets immediate EOF.

This is probably closer to what you wanted:

SearchCriteria1=$(echo "$data" | grep "Foo " | cut -c 10-)
SearchCriteria2=$(echo "$data" | grep "Foo2 " | cut -c 13-)

The semicolons on the end were useless so I took them off.

Also you probably want to append to test.1 and test.2 otherwise each iteration through the loop overwrites what the previous one wrote.

Looking at the big picture, it appears that you want a "multi output grep". Your read loop with individual greps for each line is one way to implement it, but not very efficient. Some other ways were discussed recently at https://stackoverflow.com/questions/11676350/grepping-a-20g-file-in-bash/11676853

  • Thanks for the feedback. That all makes sense but it's still not working. I'll revise my question to be more specific. – Mike B Aug 13 '12 at 4:37
input stream producer |
egrep 'Foo:|Foo2:' |
while read foo data
        case $foo in
                Foo:) echo "do something with $foo -- $data" ;;
                Foo2:) echo "do something else with $foo -- $data" ;;

The egrep line is not strictly necessary but cuts out unnecessary input for the loop. Then read into two variables (foo and data) and use case to choose between them and act accordingly (you could at this stage assign data to another variable if you wish).


How about this:

$ ./another.app | awk '/Foo: / { print $2 }' > test.1
$ ./another.app | awk '/Foo2: / { print $2 }' > test.2
  • Piping it to test.1 and test.2 is just an example of what I would do with the variable. There's still more I want to do with the variable in the script. Sorry for the confusion... the script isn't finished yet and I'm trying to keep the example straight-forward. Thanks for your patience. – Mike B Aug 13 '12 at 5:01
  • SearchCriteria1=$(./another.app | awk '/Foo: / { print $2 }')? – quanta Aug 13 '12 at 5:10

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.