13

Is there a way to run a script every minute (or 2, or 5, etc), but only if it isn't already running?

We have a set of scripts that need to run every minute. Sometimes they might start and finish in a second, other times they might go on for 5 minutes.

Our current way of avoiding simultaneous executions is by setting a is_running flag in each script, and exiting if it's still enabled. But this is a little unreliable (i.e., fatal errors would cause the flag to remain enabled even after the script halted).

We could write our own little manager, but I'm wondering if there is a more fashionable solution that already exists.

2
  • +1 I am interested in this also. I do not know the answer, however I am interested in the possible solutions. Apr 6, 2010 at 16:16
  • A minutely cron smells a lot of "make me a daemon"
    – AD7six
    Jan 28, 2015 at 8:43

5 Answers 5

12

a better way is to use flock instead of a pidfile. check the manpage: flock(1). The advantage is that no matter how a process finishes/dies, the lock is gone with it.

3

I would tend to agree with Warner's pid file answer. However, does the following feature of Anacron accomplish this?

-s
    Serialize execution of jobs. Anacron will not start a new job before the previous one finished. 

I haven't tested it myself, I don't find anacron's documentation thorough enough ...

If you want be particularly lazy ;-) You can just have the script exit if greping through ps output returns the process as running. But a lock/pid file is best.

4
  • anacron is a more recent version of cron, so it has more features than the old standard. about the 'how', it probably registers a SIGCHLD handler, which is triggered when a child process dies. thus it can keep track of running childs and simply skip those jobs.
    – Javier
    Apr 6, 2010 at 16:38
  • 1
    @ Javier: I disagree with "anacron is a more recent version of cron, so it has more features than the old standard." There is some overlap between what anacron and cron do, but they solve different problems. Using cron, how would you start a job that was missed because a machine was down? Using anacron, how would you start a job every December 25th or every 5 minutes? (You can, of course, script workarounds, but that's not what I'm talking about.) Apr 6, 2010 at 16:54
  • @Kyle: That option causes different jobs scheduled to run at the same time to run one at a time. It might work for the OP's need, but it would block other jobs or other jobs would block it. Also, anacron doesn't run at periods shorter than a day. Apr 6, 2010 at 16:58
  • @Dennis Williamson: i guess you're right, i haven't checked the history details. But still i guess you'll agree that anacron project started after cron was pretty much established, and is inspired by it; therefore it's not suprising that it has features that cron doesn't. Also, the original question was about cron
    – Javier
    Apr 6, 2010 at 17:59
1

That's the proper solution for that approach. Typically, a pid file would be used and a pid test would be done against the process to insure it's running. If stale, the lockfile would be removed and the process would run anyway.

Any additional intelligence would typically be written in the software itself, as a daemon, as opposed to running in cron.

1

If your scripts are coded in a language which supports the flock(2) syscall, then you could flock() a lock file with a function call too. Example:

If you are stuck with Bash or other scripting language which doesn't support this, the already proposed solution with flock(1) is fine too.

P.S. In any case, you should create a separate lock file only once (if it doesn't already exist), and never delete it. The directory "/var/lock" is a good place for such files.

0

This is a solution if your script runs in python (or you could create a python command to run before the next command) - I ran into this exact problem last week, and although I did find some good solutions, I decided to make a very simple and clean python package and uploaded it to PyPI. You could certainly lock __file__ to not have to think about giving it a custom resource name to lock.

Install with: pip install quicklock

Using it is extremely simple:

[nate@Nates-MacBook-Pro-3 ~/live] python
Python 2.7.6 (default, Sep  9 2014, 15:04:36)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from quicklock import singleton
>>> # Let's create a lock so that only one instance of a script will run
...
>>> singleton('hello world')
>>>
>>> # Let's try to do that again, this should fail
...
>>> singleton('hello world')
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/Users/nate/live/gallery/env/lib/python2.7/site-packages/quicklock/quicklock.py", line 47, in singleton
    raise RuntimeError('Resource <{}> is currently locked by <Process {}: "{}">'.format(resource, other_process.pid, other_process.name()))
RuntimeError: Resource <hello world> is currently locked by <Process 24801: "python">
>>>
>>> # But if we quit this process, we release the lock automatically
...
>>> ^D
[nate@Nates-MacBook-Pro-3 ~/live] python
Python 2.7.6 (default, Sep  9 2014, 15:04:36)
[GCC 4.2.1 Compatible Apple LLVM 6.0 (clang-600.0.39)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from quicklock import singleton
>>> singleton('hello world')
>>>
>>> # No exception was thrown, we own 'hello world'!

Take a look: https://pypi.python.org/pypi/quicklock

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .