I would use incremental ZFS send/receive. It should be more efficient than rsync
as ZFS knows what has been changed since the previous snapshot without needing to explore the whole file system.
Assuming you want to fully backup a file system namen datapool/fs
.
You first create a pool to store your backup on the destination server and a recursive snapshot on the source pool:
dest # zpool create datapool ...
source # zfs snapshot -r datapool/fs@snap1
then you send the whole data as an initial backup:
source # zfs send -R datapool/fs@snap1 | ssh dest zfs receive datapool/fs
Next week (or whatever period you like), you create a second snapshot on the source pool and send it incrementally on the destination on. That time, ZFS is smart enough to only send what has changed during the week (deleted, created and modified files). When a file is modified, it is not sent as a whole but only the modified blocks are transmitted and updated.
source # zfs snapshot -r datapool/fs@snap2
source # zfs send -ri snap1 datapool/fs@snap2 |
ssh dest zfs receive -F datapool/fs
Repeat the operation with incrementing the snapshot numbers each time you backup.
Remove the unused old snapshots on either servers when you no more need them.
If you have bandwidth constraints, you can compress/decompress data on the fly, for example with inserting gzip
/zip
commands in the pipeline or by enabling ssh compression.
source # zfs send -ri snap1 datapool/fs@snap2 | gzip |
ssh dest "gunzip | zfs receive -F datapool/fs"
You might also leverage mbuffer
get a steadier bandwidth usage, as described in this page:
dest # mbuffer -s 128k -m 1G -I 9090 | zfs receive datapool/fs
source # zfs send -i snap2 datapool/fs@snap3 |
mbuffer -s 128k -m 1G -O w.x.y.z:9090
Note: The zfs -r
flag is not available with non Solaris ZFS implementations, see http://lists.freebsd.org/pipermail/freebsd-fs/2012-September/015074.html . In such case, don't use the -F
flag on the target but instead explicitly rollback datasets. If new datasets have been created on the source, send them first independently before doing the snapshot + incremental send/receive.
Of course, if you have only one file system to backup without an underlying dataset hierarchy, or if you want to perform independent backups, the incremental backup is simpler to implement and should work identically whatever the ZFS implementation:
T0:
zfs snapshot datapool/fs@snap1
zfs send datapool/fs@snap1 | ssh dest zfs receive datapool/fs
T1:
zfs snapshot datapool/fs@snap2
zfs send -i snap1 datapool/fs@snap2 |
ssh dest zfs receive -F datapool/fs