51

I am running the following command every 5 minutes in my crontab to keep Phusion Passenger alive.

*/5 * * * * wget mysite.com > /dev/null 2>&1

When I run this it performs a wget on the site url routes STDOUT/STDERR to /dev/null. When I run this from a command line it works fine and doesn't produce an index.html file in my home directory.

When it runs from cron, it creates a new index.html file every five minutes, leaving me with a ton of index files which I don't want.

Is my syntax incorrect for running the cron job? From a command line it works without a problem but from cron it generates a index.html file in my home directory.

I'm sure I'm making a simple mistake, would appreciate it if anyone could help out.

1
  • 1
    Another question is why this isn't creating a file when you run it from the command line by hand. As far as I can tell from the documentation, the only difference between running wget from a terminal and otherwise is whether a progress bar is displayed.
    – Barmar
    Aug 12, 2014 at 21:42

5 Answers 5

80

You could do it like this:

*/5 * * * * wget -O /dev/null -o /dev/null example.com

Here -O sends the downloaded file to /dev/null and -o logs to /dev/null instead of stderr. That way redirection is not needed at all.

2
  • 2
    Thanks, this is more direct than redirecting to STDERR/STDOUT. I appreciate it.
    – nulltek
    Aug 11, 2014 at 13:22
  • -o /dev/null can be replaced with -q Nov 8, 2022 at 19:42
20

Do you need to actually download the contents or just receive the 200 OK? If you only have to have the server process the request, why not simply use the --spider argument?

2
  • That's a good thought. I really only need the 200 OK response.
    – nulltek
    Aug 11, 2014 at 19:32
  • I was hoping somebody unbiased would point it out, but... which solution did you end up using? My answer is really the correct way to do this :)
    – Nacht
    Aug 18, 2014 at 5:20
10

I would use the following:

/5 * * * * wget -O - mysite.com > /dev/null 2>&1

The -O - option makes sure that the fetched content is send to stdout.

2
  • 4
    Note that foo > /dev/null 2>&1 is more concisely written as foo &> /dev/null.
    – amalloy
    Aug 11, 2014 at 19:22
  • 3
    @amalloy Only in bash. In sh, which is usually what cron uses, the ampersand redirect does not work.
    – Soviero
    Aug 13, 2014 at 0:06
7

You say you only need the "200 OK" response in a comment.

That allows for solution with some additional advantages over those of
wget -O /dev/null -o /dev/null example.com. The idea is not to discard the output in some way, but not create any output at all.

That you only need the response means the data that is downloaded into the local file index.html does not need to be downloaded in the first place.
In the HTTP protocol, the command 'GET' is used to download a document. To access a document in a way that does everything except actually downloading the document, there is a special command 'HEAD'.
When using 'GET' for this task, the document is downloaded and discarded locally. Using 'HEAD' does just what you need, it does not transfer the document in the first place. It will allways return the same result code as 'GET' would, by definition.

The syntax to use the method HEAD with wget is a little odd: we need to use the option --spider. In this context, it just does what we want - access the URL with 'HEAD' instead of 'GET'.
We can use the option -q (quiet) to make wget not output details about what it does.

Combining that, wget will neither output anything to stderr, nor save a document.

wget -q --spider 'http://example.com/'

The exit code tells us whether the request was successful or not:

$ wget -q --spider 'http://example.com/'
$ echo $?
0
$ wget -q --spider 'http://example.com/nonexisting'
$ echo $?                                          
8

For a command in crontab, the fact that there is no output in both cases means you can use getting no output as an indication of errors again.

Your example command would be changed to this:

*/5 * * * * wget -q --spider mysite.com

This has the same advantages as wget -O /dev/null -o /dev/null example.com. The additional advantage is that the log output, and the document output, are not generated, instead of generated and discarded locally. Or course the big difference is avoiding to download and then discard the document, index.html.

1
  • I like this approach as well. I appreciate your feedback and answer.
    – nulltek
    Aug 17, 2014 at 19:59
3

to keep Phusion Passenger alive.

May your question should be about this, the webpage says:

A fast and robust web server and application server for

This shouldn't require any keepalive scripts.

Otherwise kasperd's solution is perfect.

4
  • Thanks for the feedback, although it isn't very constructive. Application servers do fail - although it's usually not the container's fault. Aug 11, 2014 at 17:29
  • 1
    I agree it shouldn't require any cronjobs to keep it alive. But it was a quick fix while I research Nginx/Passenger tuning. Was really just looking for the best way to output to /dev/null. I've had passenger fail or hang for 2 minutes at a time when there's no load, so requesting the url keeps passenger fired up for now.
    – nulltek
    Aug 11, 2014 at 19:41
  • 1
    It would be good to understand, what it is that is being kept alive by the wget commands. In many situations the need for keep alive messages is a symptom of an underlying design flaw which should be fixed. But even if all of those are fixed, there will still be a few cases left where a keep alive message is the right solution. Even if keep alive messages are not needed, the cron job could still be a useful part of a monitoring setup.
    – kasperd
    Aug 12, 2014 at 11:06
  • 1
    This would be better as a comment than an answer.
    – moopet
    Aug 12, 2014 at 11:34

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .