1

I have a pair of USB drives I use for backing up to via rsnapshot. One drive is left plugged in and the other is taken off site. For the first swap, I ran "rsync -aH" to copy all the contents from current drive onto the offsite drive. This process took 6 full days. I've now reached the point where I'd like to update the offsite drive.

Since rsnapshot "slides" the backups so that daily.0 is always the newest, I did the math (24 new backups since the last rotation) and renamed the existing snapshots to where they'd match.

Then I ran "rsync -aH" again, but it appears that rsync is not detecting that the existing files are linked to the same files as the newer ones and so it's done fresh copies of them. eg, even though the latest in daily.0/server/etc/passwd is the same as the existing daily.30/server/etc/passwd, they have different inodes (but daily.31/server/etc/passwd shares it's inode with daily.30 as expected).

So, on the original rsnapshot created disk currently:

daily.0/server/etc/passwd  inode: A
daily.30/server/etc/passwd inode: A
daily.31/server/etc/passwd inode: A

After the first rsync onto the off-site disk (using "current" names to hopefully make things less confusing):

daily.30/server/etc/passwd inode: A
daily.31/server/etc/passwd inode: A

And after this newest rsync:

daily.0/server/etc/passwd  inode: B
daily.30/server/etc/passwd inode: A
daily.31/server/etc/passwd inode: A

In addition to losing the hard-drive space, it's taking rsync days to verify things are in sync. It's not done yet, but I'm not going to be surprised if this process is also going to take 6 days. :-(

So, what's a better mechanism for refreshing/resyncing these drives? I'd like to stay away from wiping and re-rsyncing because of the amount of time it takes to write the data.

(One additional pain: the drives are not identical and they have other things on them, so just dd'ing them is currently not an option.)

  • Buying a USB3 or controller, and faster drives would probably be the easiest fix. Then just wipe/re-sync. – Zoredache Sep 26 '14 at 16:34
  • You used rsync the first time and rsnapshot the second time. Maybe there was a difference that made the new inodes get created. Try running rsnapshot a second time and make sure new inodes are not created for equal files. (Use a small directory as a test.) – TomOnTime Sep 27 '14 at 4:57
  • @TomOnTime No, sorry if I confused you. The data is rsnapshot created backups, I'm rsync'ing that data onto another drive. The first rsync maintained all the hard links, but then an additional rsync did not maintain the hardlinks between the new files and the old files, even though on the source all those files share the same inode. – Philippe Chaintreuil Sep 27 '14 at 13:38
  • What filesystem are you using? Some don't support inodes and fake it to the operating system. For example, MS-DOS doesn't really have inodes. – TomOnTime Sep 27 '14 at 18:18
  • They're on ReiserFS (3) drives on Linux. – Philippe Chaintreuil Sep 29 '14 at 15:02
0

I see you mentioned that it took 6 days to do the original copy over USB. If you are not using USB 3.0, then it would make a massive difference to do so. An add-on USB 3 controller is very cheap too.

  • @Zoredache mentioned this as well in a comment on another answer. Hardware does not allow for modifications, unfortunately. – Philippe Chaintreuil Nov 4 '14 at 18:51

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.