Crontab failing silently for one task but can't reproduce by running same command

Question

I have a cron job that runs a django management command that clears my mail queue. It turns out it wasn't sending, and several thousand customers had not received emails.

Any ideas what's going on here?

I have this line in my crontab (and has been the same for the past 3+ years.)

* * * * * /srv/grove_project/bin/cron.bash send_mail 

When I run it by the same user manually it works.

/srv/grove_project/bin/cron.bash send_mail

In /var/log/syslog there is

Sep 20 10:10:01 ~ CMD (/srv/grove_project/bin/cron.bash process_fulfillment_postback > /dev/null 2>&1)
Sep 20 10:10:01 ~ CMD (/srv/grove_project/bin/cron.bash send_mail )
Sep 20 10:10:01 ~ CMD (/srv/grove_project/bin/cron.bash monitor > /dev/null 2>&1)

The other commands are working fine.

What gives?

---

Update

More information: the output of the command is:

/srv/grove_project/env/lib/python2.6/site-packages/django/conf/__init__.py:75: DeprecationWarning: The ADMIN_MEDIA_PREFIX setting has been removed; use STATIC_URL instead.
  "use STATIC_URL instead.", DeprecationWarning)
INFO 2012-09-20 12:34:02,210 root 31267 ------------------------------------------------------------------------

Here's the function spitting that out:

    logging.info("-" *  72) #<--- this line printed
    # if PAUSE_SEND is turned on don't do anything.
    if not PAUSE_SEND: 
        send_all()
    else: 
        logging.info("sending is paused, quitting.")

I'm not seeing the "sending is paused" message, so looking into the send_all function

lock = FileLock("send_mail")
logging.debug("acquiring lock...") #<-- I don't see this line

Since I don't see the acquiring lock message, it must be silently hanging on FileLock only for crontab?

Answer 1

Send the output an error to a logfile. Instead of this:

* * * * * /srv/grove_project/bin/cron.bash send_mail

Do this:

* * * * * /srv/grove_project/bin/cron.bash send_mail > /tmp/cron.bash 2>&1

Any error from the command should be in /tmp/cron.bash . What does that tell you?

As a best practice, it is always a good idea to send the STDOUT and STDERR from a cronjob to somewhere. That cronjob might be spewing email to an email address somewhere (Such as 'root'). I am sure that the person who receives root's email doesn't appreciate it.

Personally, I like sending my cronjob output to syslog using /usr/bin/logger. If there is a problem, simply check /var/log/messages.

Answer 2

Append this > /tmp/send_mail.log 2>&1 to the end of your cron, then wait for it fail again, and take a look at the send_mail.log to see what happens.

---

What is the difference in environment between calling the script myself vs crontab?

Cron runs under a very minimal environment.

You can redirect your current env to a file:

env > ~/env.full

then run your job with this environment:

* * * * * env - $(/bin/cat /home/yuji/env.full) /srv/grove_project/bin/cron.bash send_mail > /tmp/send_mail.log 2>&1