Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Why is this cron job executing repeatedly over time and what can I do to stop it?

I have a cron job that is supposed to run at 4 each morning. It hits a php script that executes some daily data analysis and under normal conditions runs once (taking about 2-3 minutes to complete) and quits. It has been working but lately it is running amuck right about the time the server crashes. I investigated and found the following. The crontab entry looks like this:

* 4 * * * /usr/bin/wget -q -O /dev/null 'http://123.456.78.90/index/thing?param=1'

In my log file I see:

123.456.78.90 - - [28/Nov/2012:04:00:01 -0800] "GET /index/thing?param=1 HTTP/1.0" 200 4181 "-" "Wget/1.12 (linux-gnu)"
123.456.78.90 - - [28/Nov/2012:04:01:01 -0800] "GET /index/thing?param=1 HTTP/1.0" 200 4181 "-" "Wget/1.12 (linux-gnu)"
123.456.78.90 - - [28/Nov/2012:04:02:01 -0800] "GET /index/thing?param=1 HTTP/1.0" 200 4181 "-" "Wget/1.12 (linux-gnu)"
// and then later it ends with (note that it isn't trying every minute now)
123.456.78.90 - - [28/Nov/2012:05:28:09 -0800] "GET /index/thing?param=1 HTTP/1.0" 200 4182 "-" "Wget/1.12 (linux-gnu)"
123.456.78.90 - - [28/Nov/2012:05:29:36 -0800] "GET /index/thing?param=1 HTTP/1.0" 200 4182 "-" "Wget/1.12 (linux-gnu)"
123.456.78.90 - - [28/Nov/2012:05:29:00 -0800] "GET /index/thing?param=1 HTTP/1.0" 200 4182 "-" "Wget/1.12 (linux-gnu)"
123.456.78.90 - - [28/Nov/2012:06:06:51 -0800] "GET /index/thing?param=1 HTTP/1.0" 200 4181 "-" "Wget/1.12 (linux-gnu)"
123.456.78.90 - - [28/Nov/2012:06:06:53 -0800] "GET /index/thing?param=1 HTTP/1.0" 200 4181 "-" "Wget/1.12 (linux-gnu)"

Why would it show repeated runs like this? Is it not reaching something so it keeps trying? Any assistance is appreciated as this multiple runs thing is new to me.

share|improve this question
add comment

3 Answers

up vote 4 down vote accepted

Looks like you're running it every minute of the fourth hour of the day. So 60 copies of wget are getting fired up.

* 4 * * * .....

They then take however long they take. With 60 copies running, that's probably quite a while, explaining why some requests finish hours later.

If you really want it to run only once at exactly 4 am, then use:

0 4 * * * .....
share|improve this answer
    
Shoot... I think you are right. I was looking at that * and wondering a bit but I didn't wonder enough. –  Lothar_Grimpsenbacher Nov 28 '12 at 18:01
add comment

If you want that wget to run each day at 4am, I would suggest changing your cron job to look like this:

0 4 * * * /usr/bin/wget -q -O /dev/null 'http://123.456.78.90/index/thing?param=1'

http://www.adminschoice.com/crontab-quick-reference

I suggest fixing that first then reporting back if you're still seeing an issue. Thanks.

share|improve this answer
add comment

http://www.gnu.org/software/wget/manual/wget.html

Wget has been designed for robustness over slow or unstable network connections; if a download fails due to a network problem, it will keep retrying until the whole file has been retrieved. If the server supports regetting, it will instruct the server to continue the download from where it left off.

I'd suspect it's timing out (the default is 900 seconds) and reattempting.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.