Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I'm using rsync to backup some data from one server to another. All works fine, but it might take longer to finish depending on how much data there is to transfer.

Is there any gauranteed way to ensure that an rsync command doesn't start before the previous one finished using a cronjob?

For example, every hour I run the rsync command, but its possible the transfer takes more than 1 hour to complete, so the next one would start before the previous one finishes.

share|improve this question
    
If the job potentially takes longer than one hour to complete, and you are scheduling it closer than the duration, then you are mis-scheduling the job. Either figure out how to cut down on the time or increase the interval between jobs. If you are continually doing remote backups, you might want to consider a new disaster recovery plan. –  vgoff Dec 28 '12 at 0:31

8 Answers 8

up vote 7 down vote accepted

You can implement some kind of locking to check if rsync is still running:

pgrep -c rsync

Edited and added from the comments:

pgrep -c rsync || rsync ...

This will run the rsync command only if there is no running rsync process.

share|improve this answer
    
So I could do something like pgrep -c rsync && rsync ... ? –  chovy Dec 28 '12 at 1:36
    
Yes, you can do a pgrep -c rsync || rsync .... –  mgabriel Dec 29 '12 at 9:24
    
perfect - thanks. –  chovy Dec 29 '12 at 16:04

I couldn't get mgabriel's solution to work on OSX as the OSX version of pgrep does not seem to have the -c option (I assume this is for count). Instead I used the following:

[ $(pgrep ping | wc -l) -eq 0 ] && ping multiplay.co.uk || echo "Sorry, ping already in progress"

I used ping as an example command.

Hope this helps.

share|improve this answer

You can use the flock command to help you do this e.g. In this case flock -n is probably what you want as it will cause an immediate failure of the command if it can't obtain the lock e.g.

30 * * * *  /usr/bin/flock -n /tmp/myRsyncJob.lck /path/to/your/rsyncScript 
share|improve this answer
    
In general, predictable file names in /tmp are often dangerous due to race conditions and broad access to the /tmp directory. Is it safe in this case? –  mc0e Sep 24 '13 at 2:43

Use hatools (http://www.fatalmind.com/software/hatools/) to lock rsync cron in waiting mode.

share|improve this answer
    
Isn't this simply a wrapper around flock(1) –  WarmWaffles Oct 19 '13 at 5:56

My Answer is somewhat same what Mike told.

In the script, you should put something like this:

  • create a lock file
  • Check for the existence of the lock file when you run it next time.

But there is one very important thing you should be doing. and that to implement a trap system.

So, with that, what you can do, is that even if somehow your script is killed or someone killed it, then you can trap that signal and remove the lock file, so that you don't have a stale lock file.

You can read how to implement that over here.

Just one small thing, you can't trap signal 9, I mean if someone do kill -9, you can't trap that as that signal directly interacts with the kernel and there is no way to trap that.

Also, as suggested by John, you need to remove the lock file everytime your system reboots, just to make sure that there is no stale file left.

That you can easily do by putting a small rm -f <FILE> command in /etc/rc.local

share|improve this answer

Have a look at anacron (anachronistic cron) with the -s (serialize) switch. Serialize ensures that the command won't be called again if the previous one is still running.

share|improve this answer
    
You may have misunderstood the question. –  John Gardeniers Dec 29 '12 at 9:53
    
I don't think so. The question is "Is there any gauranteed way to ensure that an rsync command doesn't start before the previous one finished using a cronjob?" Anacron runs cronjobs with extra/different functionality. Serialize ensures that any command you call doesn't start until the previous one finishes. –  tudor Dec 31 '12 at 4:59
    
My apologies. It was me that misread the question. –  John Gardeniers Dec 31 '12 at 6:01
    
Thank you for the recognition. :-) –  tudor Jan 2 '13 at 4:25

Here is what I'd do. Create a wrapper script around the rsync to create a lock file.

script 1
- create lock file
- rsync
- remove lock file

script 2 (running later then script 1)
- check if lock file is there
    - if not run
    - if it is there wait 10 minutes in a loop. break out of lopp when the lock file is gone
- continue to run script
share|improve this answer
1  
Just be sure to also remove the lock file after a reboot, otherwise you could end up with a process that never runs again. –  John Gardeniers Dec 28 '12 at 2:07

If you're willing to consider other tools, you could also have a look at rdiff-backup. It uses librsync to do backups, and saves a configurable number of deltas/increments. It also locks so that only one rdiff-backup process can run at any given time.

share|improve this answer
    
I use rdiff-backup aswell. But you need to be careful in this setup as rdiff-backup takes more time to complete than rsync alone. –  mgabriel Dec 29 '12 at 9:26

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.