Effectively managing crontab

Question

My crontab looks something like this;

1 * * * * /var/www/cron/site1.sh > /dev/null 2>&1

0 * * * * /var/www/cron/site2.sh > /dev/null 2>&1

3 * * * * /var/www/cron/site3.sh > /dev/null 2>&1

This works great and lets me place all the nasty little script calls into one place, rather than making crontab harder to read than it already is.

But, this fails massively when site2.sh needs one script to run once a day, another to run once a week and another to run every 5 minutes. And of course it gets worse as new scripts get added with different timings.

Is there a better way?

EDIT

By better I mean making it more manageable, having a large crontab is not manageable, but neither is having scripts all over the place.

Not a GUI necessarily.

Answer 1

Making crontab more manageable, something I've definitely dreamed of many times. When it comes down to it though, if you need 20 scripts to be scheduled to run - well - you need 20 scripts to be scheduled to run.

The primary issue, at least for me, has always been creating a new entry in /etc/crontab (or in a user's crontab -e). Especially when these new entries are set to run at the same time as an existing entry (or even a list of entries).

Say, for instance, you needed 5 different scripts to run on-the-hour every-hour. You could create 5 different entries in crontab for this - or - you can take advantage of run-parts. run-parts allows you to specify a crontab entry just like any other entry, but the "command" is actually a directory. When it's the command's time to execute, it will execute every script in the given directory.

Here's a list that are common to linux systems:

01 * * * * root run-parts /etc/cron.hourly
02 4 * * * root run-parts /etc/cron.daily
22 4 * * 0 root run-parts /etc/cron.weekly
42 4 1 * * root run-parts /etc/cron.monthly

In the first line, /etc/cron.hourly states that every hour at 1-minute-past-the-hour (01 * * * *), all of the scripts that are located in the /etc/cron.hourly directory will be executed. The same concept is applied to the other lines as well. You're not limited to these times and directories either - you can fully customize them to suit your needs as a standard crontab entry.

Other than batching-scripts by directories with run-parts, you can make things more manageable by separating them by "users" and putting them into each user's crontab via crontab -u username -e. Say, for instance, you have a handful of scripts specific to reporting, you could create a "reportRunner" user and assign all report-related crons to that user. Doing this, you'll easily have list-separation and it would be easy to manage the different categories of scripts/schedules (in my opinion). It won't make the total list of crons shorter, but it should help make any-given list you're looking at shorter (and categorized).

Answer 2

I've been party to some fairly horrific crontabs and there's no getting around them. However, I've seen a few good ground rules:

1. Create structure
2. Group scripts by site/name/function
3. Annotate everything - that way, others can see what is going on.
4. Group crons by site (if you have many) eg. /var/www/site/cron/

-----8<-----

#### Site A ####

  #--- tasks ---#

    # cleanup sessions (ev. 20mins)
      */20 * * * * /var/www/cron/cleanup.script > /dev/null 2>&1

    # Garbage collection (ev. 24 hrs @ 0000)
      0 0 * * * /var/www/cron/gcoll.script > /dev/null 2>&1

    # De-dupe id's from x table (ev. 18 mins)
      */18 * * * * /var/www/cron/dedupe.script > /dev/null 2>&1


  #--- Reporting ---#

    # Generate mod reports (ev. 24 hrs @ 0400)
      0 4 * * * /var/www/cron/daily_report.script > /dev/null 2>&1

#### /Site A ####