Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I want to start regularly backup my home directory on my ubuntu laptop, machine X. Suppose I have access to 2 different remote (linux) servers that I can backup to, machines A & B. Machine X will be the master, and should be synced to A and B. I could just regularly run rsync from X to A and then from X to B. That's all I need.

However I'm curious if there's a more bandwidth effecient, and hence faster way to do it. Assuming X is going to be on residential style broadband lines, and since I don't want to soak up the bandwidth, I would limit the transfer from X. A and B will be on all the time, however X, will not be, so I'd also like to reduce the amount of time that X is transfering, potentially allowing A and B to spend more time transfering. Also, X won't be connected all the time.

What's the best way to do this? rsync from X to A, then from A to B? Timing that right could be troublesome. I don't want to keep old files around, so if I was to rsync, then the --del option would be used. Could that mean something might get tranfered from A to B, then deleted from B, then transfered from A to B again? That's suboptimal.

I know there are fancy distributed filesystems like gluster, but I think that's overkill in this case, and might not fit with the disconnected nature.

share|improve this question

6 Answers 6

There are clever ways to do this, and simple ones. I'd opt for simple whenever possible.

Could you do X -> A, and then A -> B?

There are also methods like Unison, but that seems to only go to one replica, and it is bi-directional which seems to not be as useful for a backup. However, if both A and B ran things on Unison, it seems like client X could then back up to A or B, and it would make it to the other nicely.

share|improve this answer
    
Unison will work just fine if you share the directory over 3 systems, just like asked. However, unison takes into account updates on both systems, meaning that if you did some updates on X, and some updates on A since the last sync, X will get the new updates from A and A will get the new updates from X. But if you only update X, and never the backups A and B, it will work just fine. Also, Unison, just like rsync, is very efficient regarding bandwidth usage. –  rubenvdg Mar 11 '10 at 19:54

You want X to initiate the sync with A, which will then pull the data from X and when finished push to B. A script on A could just check for the existence of a file somewhere, that is uploaded to A by X when it is online and ready for a sync. When it is done pulling from X, it should push to B and when finished remove the file. Alternatively you could have almost the same script on both A and B, but when A is done pulling from X, it puts a file on B telling it to sync with A. If you had more than 2 machines to sync, this would be the way to go since then the sync can then just cascade through all of them (possibly in a distributed tree fashion rather than linear)

share|improve this answer

Would Dropbox work for you?

share|improve this answer

gibak is right for this, I think.

You can have the laptop order the servers to pull from each other, then push to whichever server; though that means the laptop has to wait and possibly stay online longer. You can add cronjobs so the server pull from each other. Or you could have the laptop push to A, and A push to B in a post-receive hook, which is simpler.

share|improve this answer

I think Michael Graff's solution is the best idea. rsync X -> A. Then A -> B.

If you want another suggestion that would cut down computationation time, you could look at xdelta (http://xdelta.org/). It essentially the same solution as the multi-hop rsync, with the benefit it could save you the time of having to calculate the delatas at each hop. If the files involved are large, this could become a material savings of time.

Jim C. http://www.kleobackup.net

share|improve this answer

I agree with naelyn, this sounds like a scenario for dropbox

As you say you are running Ubuntu. I thought Ubuntu One was supposed to provide this functionality too. I don't use it myself but might be worth a look.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.