Currently files are backed up from one external hard drive to another as a means to provide the ability to restore from a secondary device should the primary external hard drive become unavailable e.g. hardware failure. The current method is predicated on the use of Microsoft Windows.
All backup is to be performed from Linux devices running Ubuntu 14.04 going forward.
All external hard drives are encrypted using TrueCrypt and have been formatted with NTFS since they are also used on other operating systems other than Linux such as Microsoft Windows.
Files are also created on Microsoft Windows and stored on these devices.
File sizes vary from as little as 100KB to as large as 10GB. There are approximately 1 million files and this continues to grow on a regular basis.
- Performance on external hard drives. All current drives are USB 2
- Speed of transfer
- Compute thresholds or considerations. The devices that currently perform the backups are either i3, i5, i7 with a minimum of 8GB of RAM
- Preserving permissions and the like
- Deep paths. Some paths traverse as deep as 20 or 30 hierarchies
- Minimal impact on the lifespan of the external hard drives
- Avoidance of changing filesystems if possible
- Whole copies are fine and deltas or differentials are not required
- Checksums to ensure that files have been successfully committed
- The ability to resume transfers in the event of a failure
Now there have been mention of using switches or options such as
inplace however there is very little information available on additional considerations for encrypted drives that leverage NTFS filesystems.
The current attempted command as an example is;
rsync -vrlptg /media/drive1 /media/drive2
What other switches/options must be included to support the requirements?
rsyncdoesn't handle changes to large files quickly - for example, moving a file within a file system. There's a good discussion of
rsyncvs ZFS replication here: slashdot.org/story/15/12/22/026209/…