Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have scheduled a cron job to run every minute but sometimes the script takes more than a minute to finish and I don't want the jobs to start "stacking up" over each other. I guess this is a concurrency problem - i.e. the script execution needs to be mutually exclusive.

To solve the problem I made the script look for the existence of a particular file ("lockfile.txt") and exit if it exists or touch it if it doesn't. But this is a pretty lousy semaphore! Is there a best practice that I should know about? Should I have written a daemon instead?

share|improve this question

7 Answers 7

up vote 31 down vote accepted

There are a couple of programs that automate this feature, take away the annoyance and potential bugs from doing this yourself, and avoid the stale lock problem by using flock behind the scenes, too (which is a risk if you're just using touch). I've used lockrun and lckdo in the past, but now there's flock(1) (in newish versions of util-linux) which is great. It's really easy to use:

* * * * * /usr/bin/flock -n /tmp/fcj.lockfile /usr/local/bin/frequent_cron_job --minutely
share|improve this answer
    
lckdo is going to be removed from moreutils, now that flock(1) is in util-linux. And that package is basically mandatory in Linux systems, so you should be able to rely on it's presence. For usage, look below. –  jldugger Apr 9 '12 at 21:44
    
Yeah, flock is now my preferred option. I'll even update my answer to suit. –  womble Apr 10 '12 at 6:22

Best way in shell is to use flock(1)

(
  flock -x -w 5 99
  ## Do your stuff here
) 99>/path/to/my.lock
share|improve this answer
    
I can't not upvote a tricky use of fd redirection. It's just too arcanely awesome. –  womble Nov 9 '09 at 11:57
1  
Doesn't parse for me in Bash or ZSH, need to eliminate the space between 99 and > so it is 99> /... –  Kyle Brandt Nov 9 '09 at 12:36
    
@Kyle Yup, correct! Fixed! @womble Haha, agreed :) –  Philip Reynolds Nov 9 '09 at 13:51
    
This is brilliant simplicity. –  sparks Nov 9 '09 at 15:08
    
beautiful! and @wombie, it's documented in flock's man page –  Javier Nov 9 '09 at 15:09

Actually, flock -n may be used in stead of lckdo*, so you will be using code from kernel developers.

Building on womble's example, you would write something like:

* * * * * flock -n /some/lockfile command_to_run_every_minute

BTW, looking at the code, all of flock, lockrun, and lckdo do the exact same thing, so it's just a matter of which is most readily available to you.

* Since, given my reputation at the time of writing, I can neither edit nor comment on previous answers, I have to write this as a separate answer.

share|improve this answer

You can use a lock file. Create this file when the script starts and delete it when it finishes. The script, before it runs its main routine, should check if the lock file exists and proceed accordingly.

Lockfiles are used by initscripts and by many other applications and utilities in Unix systems.

share|improve this answer
    
this is the only way I've ever seen it implemented, personally. I use on as per the maintainer's suggestion as a mirror for an OSS project –  warren Nov 9 '09 at 11:48

This might also be a sign that you're doing the wrong thing. If your jobs run that closely and that frequently, maybe you should consider de-cronning it and making it a daemon-style program.

share|improve this answer
2  
I heartily disagree with this. If you have something that needs to run periodically, making it a daemon is a "sledgehammer for a nut" solution. Using a lockfile to prevent accidents is a perfectly reasonable solution I've never had a problem using. –  womble Nov 9 '09 at 11:53
    
@womble I agree; but I like smashing nuts with sledgehammers! :-) –  wzzrd Nov 9 '09 at 15:01

Your cron daemon shouldn't be invoking jobs if previous instances of them are still running. I'm the developer of one cron daemon dcron, and we specifically try to prevent that. I don't know how Vixie cron or other daemons handle this.

share|improve this answer

You havent specified if you want the script to wait for the previous run to complete or not. By "I don't want the jobs to start "stacking up" over each other", I guess you are implying that you want the script to exit if already running,

So, if you don want to depend on lckdo or similar, you can do this:


PIDFILE=/tmp/`basename $0`.pid

if [ -f $PIDFILE ]; then
  if ps -p `cat $PIDFILE` > /dev/null 2>&1; then
      echo "$0 already running!"
      exit
  fi
fi
echo $$ > $PIDFILE

trap 'rm -f "$PIDFILE" >/dev/null 2>&1' EXIT HUP KILL INT QUIT TERM

# do the work

share|improve this answer
    
Thanks your example is helpful - I do want the script to exit if already running. Thanks for mentioning ickdo - it seems to do the trick. –  Tom Nov 11 '09 at 11:36

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.