1

I'm trying to set up two separate cron jobs that access the same static file. One of the jobs runs every minute, aggregating new data that has arrived into the file. The second job runs every 30 minutes, parsing the aggregated data and truncating the file to reset it for the next 30 minutes.

Obviously there's a potential here for deadlock every 30 minutes. Is there any way to write the cron job for the first task so that it only executes every minute for 29 minutes (short of listing out every minute 1-60, excluding 30 and 60), then starts again on minute 31? Thanks!

Edit

By request, I'm using PHP in order to perform these operations (since there is some significant use of data structures involved). I have implemented the use of PHP's flock function, and while the function's documentation specifies that the call blocks until 1) the lock is obtained, or 2) the LOCK_NB option is specified, I was confused by the fact that the following example seems to imply it's possible to not obtain the lock, even with a blocking call.

Unless this is due to some other issue (permissions, OS interrupts, etc)? This issue of blocking until obtaining a lock was really the motivation for my question, to see if there was another way to avoid locking files entirely, thereby mitigating the risk of a non-blocking lock attempt fail and the script halt.

2
  • Edited my comment. Remember, it's not LOCKING, it's SEMAPHORE. Locking has the idea of permanence in the name: this thing is locked, and if you need to get in it, you're out of luck. That's not correct in the context of file locks. File locks aren't permanent, they're just like a stop light, telling you to wait a few seconds before you continue. Nov 6, 2009 at 1:58
  • Ahhh, classifying it as a semaphore instead of a lock (e.g. mutex) clears it up significantly. Thank you muchly! :)
    – Magsol
    Nov 6, 2009 at 5:38

2 Answers 2

2

You don't want to do that; the one time your job takes longer than a minute, you're going to be hosed. Race conditions like that make for ugly problems: intermittent, unpredictable, and hard to diagnose.

Pboin is exactly right: you need to explore the use of semaphore in your code, and then the problem just goes away, and you don't have to worry about schedule conflicts at all.

What language are you using?

Edit (apologies for the php, haven't used it in a while, and am a little drunk, so may not compile):

The "usual" tactic with locking is to check for a lock (which in this case I do by trying to get one), and if there IS one, to implement a short wait and try again. You can try using php's sleep() function inside a while loop, e.g:

while($x){
    if(flock($file_handle, LOCK_EX)){
        $x = false;
    }else{
        sleep(5);
        $x = true;
    }
}

I use something similar to this for tons of cron-based write jobs. Works like a charm. It's actually a similar method to the ALOHA communications protocol, which is very similar to a later network protocol called "TCP/IP". Try, try, and try again.

2
  • Edited my post with the specifics of my situation, including the language I'm using.
    – Magsol
    Nov 5, 2009 at 23:12
  • Hahaha, it looks solid, though the "compile" comment regarding PHP was kind of a giveaway ;) Right, this is somewhat similar to TCP's back-off algorithm (though this is only a constant wait time, rather than quadratic), but isn't the loop and call to sleep redundant based on the fact that this is exactly what flock() does? I suppose that's my main question - is this what flock() already does, or do I need to implement this loop myself?
    – Magsol
    Nov 6, 2009 at 2:41
1

Well, you could list out your minutes in cron as you suggest, but that's not really solving the problem. (For one thing, cron doesn't guarantee execution within any given minute.)

You'd probably be better off implementing some locking functionality in your script -- it's much less of a hack. The Advanced Bash Scripting Guide, an excellent and free resource suggests the lockfile command, which will help you do this.

You'll have a lot less to worry about by doing this correctly.

1
  • Posted an edit to my post regarding this, since Satanicpuppy below also requested it.
    – Magsol
    Nov 5, 2009 at 23:11

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .