I've been tasked with making an off-site backup work over the WAN. Both storage boxes are FreeBSD based NAS boxes running ZFS.
Once or twice a week, 15-60 gigs of photography data gets dumped to the office NAS. My job is to figure out how to get this data off-site as reliably as possible using the VERY SLOW DSL connection (~700Kb/s upload). The receiving box is in much better shape, at 30Mb/s down, 5Mb/s up.
I know, carrying a hard drive off-site would move data much more quickly, but it's not an option in this case.
My options seem to be either:
- ZFS incremental send over ssh
- Rsync
rsync is a time honored solution, and has the all-important ability to resume a send if something gets interrupted. It has the disadvantage of iterating over many files and not knowing about dedup.
ZFS snapshot sending might transfer a bit less data (it knows a lot more about the file system, can do dedup, can package up the metadata changes more efficiently than rsync) and has the advantage of properly duplicating the filesystem state, rather than simply copying files individually (which is more disk intensive).
I'm concerned about ZFS replication performance[1] (though that article is a year old). I'm also concerned about being able to re-start the transfer if something goes down - the snapshot capability doesn't seem to include that. The whole system needs to be completely hands-off.
[1] http://wikitech-static.wikimedia.org/articles/z/f/s/Zfs_replication.html
Using either option, I should be able to de-prioritize the traffic by routing it through a specified port, then using the QOS on the routers. I need to avoid a major negative impact on users at both sites during each transfer, since it will take several days.
So... that's my thinking on the issue. Have I missed any good options? Has anyone else set something similar up?