I'm using ZREP to replicate two servers with each other and each server contains one ZFS-pool containing two datasets as replication master and two sets as replication target. The master sets contain the system and VirtualBox-VMs of the local server, the replication targets the same from the other one.
Additionally I'm backing up all master sets per server to some NAS using
rsync. The NAS is pretty slow and backup takes hours to succeed, so the implemented approach is to suspend VMs, create one snapshot, restore the VMs and let
rsync run from the created snapshot. The important thing is that the manually created snapshot didn't follow the ZREP naming convention and has been destroyed directly after
rsync finished again. At first, ZREP continued to operate concurrently, started by
But from time to time it happened that ZREP got into some state not able to sync anymore. To resolve that issue, a coworker told me that he needed to delete snapshots and follow the process to initialise ZREP again all over. That problem got fixed by not let ZREP run in parallel with
rsync and our own snapshot anymore in the end.
Sadly I lack the concrete details of that error and the coworker is not available anymore, but from his description it sounded like there was a problem with finding common ancestors of snapshots between replication master and target to sync incrementally. I think the error messages were something like the following:
cannot receive incremental stream: most recent snapshot of zfs-pool/vbox/tori does not match incremental source cannot open 'zfs-pool/vbox/tori@zrep_0001b7': dataset does not exist
My understanding of how incremental send/receive works.
From my understanding of the docs and other questions, to successfully send snapshots incrementally, the sending master and receiving target need to share that one snapshot which is used as argument 1 to
zfs send and that snapshots needs to additionally be the current one on the receiving target.
The second argument is an arbitrary newer snapshot, used by ZFS to calculate differences to the one snapshot master and target have in common and send those differences to the replication target. Because both share the same snapshot specified as argument 1, the differences make sense to the target and can simply be applied as is.
-I from my understanding either lead to one logical snapshot getting send containing all calculated incremental data of the master side OR sending all intermediate snapshots containing their incremental changes. So e.g.
-i leads to ONE new snapshot on the target always vs.
-I might lead to N additional ones.
Creating and destroying intermediate snapshots between what gets provided as arg 1 and 2 to
zfs send -i shouldn't be any problem, because ZFS always calculates differences only between those two provided arguments and doesn't care about any other intermediate snapshots. In case of ZREP that means in theory that as long as I'm not interfering with ZREP-managed snapshots, it shouldn't make any difference for that if additional snapshots get created during its operation or not. Simply because special ZREP-snapshots are available always, managed by ZREP and used to calculate differences for replication. So in theory, additionally creating snapshots for
rsync and backup shouldn't be a problem at all.
Are those assumptions correct?
Non-ZREP related questions.
Is it safe in general to send ZFS-snapshots incrementally ignoring some intermediate ones? Or is it necessary to send ALL intermediate snapshots created ever to the replication target to net get out of sync or stuff? How do things depend on