up vote 0 down vote favorite
1
share [g+] share [fb]

I have a 10TB XFS filesystem on a SLES 10.2 server. I am currently running rsync on it once per day, to sync it with a disk-based online backup on another server. The primary volume is quite busy as it is, and so rsync is only running at 2MB/sec on average. However, I can scp a file and get >50MB/sec to the same destination. This leads me to believe that rsync is overburdened with deciding what to transfer, not actually transfering.

What other ways can I achieve this type of synthetic full backup? What are the relative pros/cons of the following ideas:

  1. Use xfsdump, piped through ssh or similar, to xfsrestore on the destination. Can it do incrementals when used in this way?

  2. Some of sort of asyncronous block replication using drbd or similar.

  3. Some better use of rsync such as looser comparison criteria. I'm currently using

    rsync -aqAX --numeric-ids --delete-during --files-from=foo ...

link|improve this question
67% accept rate
why the --files-from? not sure, but it could force the interaction between source and target to become more linear, and less efficient when deciding if a file should or shouldn't be transferred. – Javier Oct 19 '10 at 23:00
rsync might be IO bound reading the disk, if that's the case getting a better transfer rate over the wire isn't going to do any good. Best to find the bottleneck before resorting to more drastic measures. – Chris S♦ Oct 19 '10 at 23:25
Chris S: Wire speed is certainly not the problem. I suspect the problem is the way the incremental filesystem diff is computed. – lukecyca Oct 19 '10 at 23:54
feedback

1 Answer

up vote 1 down vote

First of all I suggest there is probably a way to make rsync handle this better. Have you tried turning off deltas / partial file transfers? Only matching by time?

Secondly, skip the bit about xfsdump etc. Even if you get something working along those lines it is going to be fragile and will be a restore nightmare.

Lastly, because of the size of your data set and the problems you're hitting, I suggest looking into the use of LVM volumes. By putting your data inside an LVM volume you would get the ability to snapshot it without taking anything offline, then rsync/backup the snapshot at your leisure. This would free up a lot of file locking issues and avoid problems with data inconsistency. It's also likely there are tools related to using LVM snapshots for backup that are more efficient than rsync because they make use of lower level knowledge about the file system. Something to look into.

link|improve this answer
We're using XVM instead of LVM (it's a fully supported stack from SGI) and we've been looking at doing host-level snapshots like you describe. It's tricky because the host is actually in a failover pair, and we couldn't find a way to do redirect-on-write snapshots in such a way that failover could still happen cleanly. Thanks though—we'll give it more thought. – lukecyca Nov 2 '10 at 19:25
Hum. Yes backup up anything that is hot-swappable with other live equipment is a tricky task. Good luck with the details. – Caleb Nov 2 '10 at 21:17
feedback

Your Answer

 
or
required, but never shown
discard

Not the answer you're looking for? Browse other questions tagged or ask your own question.