Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Running macosx 10.6.2, I am seeing some extremely weird behavior with a script which repeatedly calls curl -o (file) and then greps for a certain string within it. Occasionally grep returns 1 (not found) when I'd expected 0 (found). Here's the script...

# do this 1000 times

for ii in `cat count.txt`; do

        rm -f a.txt
        rm -f e.txt
        curl  --fail --stderr e.txt -j -o a.txt  -s  $MYURL

        if [ -e a.txt ] ; then

                #  Occasionally a.txt doesn't finish writing on time
                grep  "login-url" a.txt >/dev/null
                echo $LASTERR is lasterr grep 1
                if [ "$LASTERR" -ne "0" ] ; then
                        cp a.txt anomaly.txt
                        sleep 1
                        echo "Sleeping..."

                grep -q "login-url" a.txt >/dev/null
                echo $LASTERR is lasterr grep 2
                if [ "$LASTERR" -ne "0" ] ; then
                    echo "Dying..."
                    exit 1

                # This is what I actually want to do
                grep "login-url" a.txt  >> out.txt


What I see is something like this:

0 is lasterr 1
0 is lasterr 2
0 is lasterr 1
0 is lasterr 2
0 is lasterr 1
1 is lasterr 2

In other words, a.txt is changing (as far as grep can tell) between the two greps!

Has anyone else seen the like?

I note if I put in a "sleep 1" after the curl call, the issue goes away. So is there a problem with re-using the same file name over and over, or is curl returning before it is done writing, or...?

This is not a crisis issue because of the "sleep 1" workaround, but I am nervous because I don't understand the behavior.

share|improve this question
This isn't your problem, but you should do for ii in {1..1000} or for ((ii = 1; ii <= 1000; ii++)) instead of using a file like that. Or if this isn't Bash, do for ii in $(seq 1000) or for ii in $(jot 1000) – Dennis Williamson Feb 9 '10 at 19:19

You might try making the two grep commands match. One has -q and the other doesn't. That's probably not the problem, but it's worth eliminating differences to narrow things down.

Your output doesn't match what the script is doing, so it makes me wonder if you're running and looking at two different versions. I've done that a number of times myself, causing a great deal of confusion until I realize what's going on.

share|improve this answer
From the man page of grep: Normally, exit status is 0 if selected lines are found and 1 otherwise. But the exit status is 2 if an error occurred, unless the -q or --quiet or --silent option is used and a selected line is found. In this case you are getting different results because of the -q – tajh Feb 10 '10 at 0:02
Hi, thanks for noticing that; I was a little clumsy in cleaning out the company-specific stuff from the script. Naturally, that's where the error actually showed up. I had had something like this: SERVER=(some ip address) curl -o a.txt -s http://${SERVER}/.../a.jsp?val1=a&val2=a@b&val3=c When I changed it to URL="http://(some ip addr)/.../a.jsp?val1=a&val2=a@b&val3=c" curl -o a.txt -s $URL It worked fine. I think I must have had shell-active characters in my URL... – Akaioi Feb 10 '10 at 15:58

have you tried the debug output of bash?

bash -x /path/to/script

It will output each step verbatim.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.