Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

We're using cron to manage our backups and other jobs in multiple locations. Using chef to populate files in cron.daily, cron.hourly, etc has worked pretty well for us so far, but with some issues:

  • I don't want to have to manage a mailserver on the system just to receive cron output
  • I want to be able to put output in my cron jobs without receiving email about them if nothing went wrong
  • I don't want to have to check /var/log/messages to see if jobs failed without output
  • I don't want to have to log in to the system to find that the backup job is still running

Optimally, I'd like a web-based frontend that I can use to see this information, either as an extension on cron or a complete replacement.

I can solve the above problems myself with a bit of scripting, but I'm sure that this is a problem that others have solved already.

Note that I acknowledge that this is a completely separate issue from verifying the backups after they've been completed.

share|improve this question
    
SSMTP is a nice lightweight mail daemon. You might want to check out this article. He provides a nice template that captures the full details of a script run to a file, and provide the option to email an error if it happens. It also seems kind of odd, that you don't want a mail daemon, but you are fine with a web server? –  Zoredache Aug 30 '11 at 0:01
    
It's a comfort level thing - I've configured many more web servers than I have mail servers. Also, our domain's mail setup is a mess .. I'll sort it out one day, but probably not today. –  Michael Pearson Aug 30 '11 at 0:08
    
What OS? I could suggest Windows Task Scheduler, as you've neither specified an OS nor excluded the possibility of changing to a different one to achieve your goals. –  John Gardeniers Aug 30 '11 at 3:09
    
Whups. Sorry, assumed that 'cron' implied 'generic Unix' (in this case Ubuntu 10.04) –  Michael Pearson Aug 30 '11 at 3:58
add comment

4 Answers

up vote 1 down vote accepted

You can use Resque - with it's built in web interface - to manage jobs. Check out the Resque Railscast to see it in action.

Then use the ruby gem resque-scheduler to add jobs to queues at scheduled times.

There's a good blog post on it: "Cron tasks for your Rails application with Resque"

share|improve this answer
    
Thanks Chris - I'm mainly going with this answer because I have an existing Ruby / Rails / DelayedJob stack .. and migrating the DJ jobs to Resque and having the backups, etc, managed by One Thing seems like a good idea. –  Michael Pearson Aug 30 '11 at 2:45
add comment

Maybe this isn't a perfect fit for what you're after, but my approach most recently to this has been the combination of two Ruby gems and a web service.

Firstly, the gem whenever, which gives you a clean ruby interface for scheduling cron jobs (so, not a cron replacement, just a layer over the top).

Next is the backup gem, which provides a solid set of tools for managing backups quite neatly (including to cloud storage locations). Backup will also email you when tasks fail and/or succeed (it's up to you to configure).

The final piece of the puzzle is SendGrid - and while they push their paid plans, they do have a free plan that allows for 200 emails a day. I'm certainly not coming close to taxing their servers for this script, so it suits me fine. It may take a few hours for them to confirm an account, but once it's set up, I've had no problems.

Happy to share some example scripts if you'd like.

share|improve this answer
add comment

It sounds like what you want is not a cron replacement, but rather a backup system (like Bacula).
See my answers to your individual points below.

I don't want to have to manage a mailserver on the system just to receive cron output

You don't have to with cron. As others have pointed out you can run SSMTP, or a send-only sendmail instance (just a queue runner).
Alternatively you can redirect your cron output to a file, or syslog, or pipe it off to any other program you want to handle it.

I want to be able to put output in my cron jobs without receiving email about them if nothing went wrong

Per the above, you can pipe your cron output to a handler program, logger, etc. - You don't HAVE to receive cron mails, that's just the way it's configured by default...

I don't want to have to check /var/log/messages to see if jobs failed without output

This depends on how your backup software works (and how amenable it is to producing log entries or parseable output so you can feed it through a script and send messages as needed.
A "real" backup system will provide this information for you via its console/monitoring tools, and can even email you if it needs attention during a job.

I don't want to have to log in to the system to find that the backup job is still running

Again, this depends more on your backup software. Bacula and other dedicated backup programs let you monitor the status of agents on each client machine from a central console, so you don't need to log in to the individual clients to check on their status.
Bacula has a web-based interface, though I can't vouch for it: I use the command-line console program.


None of this really helps with your "other jobs" (as you pointed out, their output can be handled with my notes from the first two points plus "a bit of scripting"), but using a standard, well-known piece of software like cron will make maintenance easier in the long term.

share|improve this answer
    
Great answer, thanks! My worry with something like Bacula is that by 'backups' I mean 'incremental create-only rsync of a gigantic directory' and 'daily mysqldumps', both over international non dedicated links. So traditional backup software like Amanda wouldn't seem like a great fit. –  Michael Pearson Aug 30 '11 at 1:16
    
And yes, I favour sticking with cron and adding a reporting layer on top of it - cron, for the most part, "just works". –  Michael Pearson Aug 30 '11 at 1:18
    
Both Bacula and Amanda can do incremental backups: They'll copy whole files when they change, but they only pick up the changed files (mtime or checksum changed). This isn't as efficient as rsync if parts of a file change, but it's a good fit for new files getting created. Database backups are "special" (you can do dumps, or quiesce a slave and back up the raw DB files -- I favor the latter) –  voretaq7 Aug 30 '11 at 1:24
add comment

You could try a structured job scheduler to manage tasks across your systems. Something like the Open Source Job Scheduler would work well for this. The benefit is that you'd have a central console to manage things, have a method to handle dependency-checking and a proper way to re-run or start jobs properly in the case of a missed job window.

share|improve this answer
    
Unfortunately looks like overkill for my needs, and more setup / configuration time than rolling my own solution. Laziness is a motivator here. –  Michael Pearson Aug 30 '11 at 0:27
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.