Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

We have a directory of files (about 4GB) - and we'd like to synchronise between 3 locations across the internet. However, we're very conscious about bandwidth, so we need the following features:

  • Schedule when synchronising should occur (e.g. 6pm - 8am)
  • Specify a speed limit (e.g. 100KB/s)
  • Don't synchronise when certain applications are running

We also need the following:

  • Works on Windows and Linux
  • Is free or not too expensive

Is there such an application in existence? We ruled out Dropbox because it's not free over 2GB, and Live Mesh because we can't set a speed limit or schedule.

share|improve this question

5 Answers 5

up vote 8 down vote accepted

Unison is close in that it will do bi-directional copying and runs on windows and Linux and is under the GPL. Unison is based on rsync so it can transfer pretty efficiently.

It doesn't implement any of the scheduling or bandwidth limiting features you want. Those other features could be implemented by combining unison with additional tools and/or scripts to automate it.

share|improve this answer
1  
+1 unison for bidirectional change detection –  Tobu Feb 1 '10 at 16:43
    
Havent tested this yet, but will accept as it seems like the best answer. –  nbolton Mar 1 '10 at 18:34

Rsync if you are using Linux/Unix.

For a Windows system you can use SyncToy from the PowerToys kit, or use Robocopy (from the Resource Kit for 2000-2003, incl. in 2008+ though) if you want to script it.

I believe there are a few versions of rsync for windows. Since you want a multi-platform sync tool, rsync may be your best bet.

One comment I would have is that this will require 3 separate paths to sync using a a tool like Rsync, and you could have some replication delays you aren't expecting. One way to mitigate this would be to have a "master" copy, and only replicate from/to that.

If you have the possiblity of using just Windows at all 3 sites for your sync needs, ultimately your best option might be DFS. You could setup VPNs using PPTP and configure a DFS root that replicates between 3 servers in the 3 sites. The new DFS-R protocol is supposed to be very efficient (similar to Rsync), and all your replication issues can be handled by the DFS engine.

share|improve this answer
    
Rsync isn't automatically bi-directional is it? –  Russell S. Pierce Jul 19 '12 at 21:18
    
Not that I know of, but I'm not an rsync expert. You'd need to rsync newer files only in both directions I think. However, DFSR will do two-way sync, but you'll want to have a monitoring solution in place to alert you on errors since DFSR is a service that runs continuously. –  Ryan Fisher Oct 27 '12 at 5:47

JungleDisk -- you get 5GB for "free" (use the Rackspace Cloud, not S3, to avoid transfer charges) with their Desktop Sync subscription that runs all of $3 a month.

http://www.jungledisk.com/personal/desktop/features/

You can set up the client software on any platform and have "sync" folders that will live push/pull (using the cloud as the go-between) to your machines. $0.15 per GB after 5 so it's really cheap if you go over.

The product is owned/backed by Rackspace, a very reputable company (so it's not going to disappear overnight).

share|improve this answer
    
I used to use Rackspace, they're spot on. I had no idea they offered this product. –  nbolton Jan 20 '10 at 18:46
    
I should add they do the bandwidth throttling and timing you're looking for as well, everything is nicely integrated into the software and is very easy to install and manage - while I'm a die-hard fan of rsync this is just a nice offering that wraps it all up for you for less than the price of one sammich a month. I use it to backup my home laptops & such. –  troyengel Jan 20 '10 at 19:26

If you can, I would seriously consider using Dropbox for synchronization. It would be well worth the $99/yr to not have to worry about synchronization, failures, disk space problems, etc. It doesn't do the scheduling thing you asked about but is always-on, but you can throttle it. It does bit-level syncing also saving bandwidth. Depending on the funding of your project I would go with that if at all possible and just worry about your actual project and files.

rsync would work well, but you will need to set up all of the scripting and you will have occasional errors and have to monitor it to make sure everything is syncing right. I use this as well but it stops working randomly and I have to fix a little bit here and there to keep it running.

A program I use but it Windows-only is SyncBackSE. It does all that you mention, including bandwidth throttling and scheduling. I've used it at home for backups for a number of years and have been very happy with it.

Since you have 3 hosts these systems, other than Dropbox (which I believe just keeps the latest file), may have problems with it. What if a file changes on 2 more more hosts?

share|improve this answer

Rsync also has a windows version. It runs as a batch file you can schedule in your scheduled tasks. Works like a charm

share|improve this answer
    
Deltacopy is a port/wrapper that works reasonably well in Windows. –  Russell S. Pierce Jul 19 '12 at 21:19

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.