How to efficiently backup shares (ZFS). Folder structure might change but media usually do not

Question

The Issue: I have 18 large shares (each 24TBs), that contain lots of big media files. Mostly the media files themselves stay untouched. However, sometimes the folder structure changes, and thus my implementation of rsync forces TB's of data to be deleted and then re-copied.

rsync -avrXn --delete-before --progress --exclude 'zzToTRASH' /mnt/Storage3_Live/Sirius/ /mnt/Storage3_bak/Sirius_bak/ (I have to delete before in order to not run out of space on the target)

The Goal: I want to make a mirror (for on the shelf disaster backup) and every 3-4 months want to make an incremental back up so the new folder structure is preserved but without re-copying a lot of the media files themselves. I am reading about hard links in rsync and cp, but.. I am still not clear which is the most reliable method.

Caveats:

• I want to preserve extended attributes.
• Running FreeBSD XigmaNAS, FreeBSD 11.4-RELEASE-p7 #0 r369206M: Wed Feb 3 00:07:44 CET 2021

Answer 1

Jeez. It's like using a microwave oven as a fuel for campfire in the kitchen.

1. read the docs at least, you're completely ruining the whole zfs concepts. if you'll chose to skip that point - then merely stop using zfs, it's counterproductive this way: consumes large amounts of RAM and extra disk space with no additions benefits.

2. start using zfs snapshots: this will make the 1st tier of your backups and protect you from the damage that can occur on the logical level (files being deleted or incorrectly modified).

3. back up using incremental snapshot transfer between two zfs-enabled hosts. this will protect you from losing data if physical disk corruption will occur.

3a) monthly backups, really ? use daily or hourly incremental snapshot transfer.