Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

We have an external USB 2 drive that we are using as a backup drive for our configuration. We use rsnapshot for the backups. It uses a few standard commands for managing snapshots:

  1. rm -rf: deletes expired snapshots
  2. mv: moves older snapshots down a slot
  3. cp -al: duplicates last snapshot to new slot
  4. rsync -a --delete --numeric-ids --relative: synchronizes new snapshot

As you could see by the log below, the majority of the time is spent on the rm -rf and the cp -al steps:

[25/Dec/2010:14:00:02] rsnapshot hourly: started
[25/Dec/2010:14:00:02] echo 21012 > /var/run/rsnapshot.pid
[25/Dec/2010:14:00:02] rm -rf /mnt/extdrive/snapshots/hourly.5/
[25/Dec/2010:14:15:48] mv /mnt/extdrive/snapshots/hourly.4/ /mnt/extdrive/snapshots/hourly.5/
[25/Dec/2010:14:15:48] mv /mnt/extdrive/snapshots/hourly.3/ /mnt/extdrive/snapshots/hourly.4/
[25/Dec/2010:14:15:48] mv /mnt/extdrive/snapshots/hourly.2/ /mnt/extdrive/snapshots/hourly.3/
[25/Dec/2010:14:15:48] mv /mnt/extdrive/snapshots/hourly.1/ /mnt/extdrive/snapshots/hourly.2/
[25/Dec/2010:14:15:48] cp -al /mnt/extdrive/snapshots/hourly.0 /mnt/extdrive/snapshots/hourly.1
[25/Dec/2010:14:23:32] rsync -a --delete --numeric-ids --relative /etc /mnt/extdrive/snapshots/hourly.0/sm4/
[25/Dec/2010:14:23:52] touch /mnt/extdrive/snapshots/hourly.0/
[25/Dec/2010:14:23:52] rm -f /var/run/rsnapshot.pid
[25/Dec/2010:14:23:52] rsnapshot hourly: completed successfully

My questions:

  1. I'm currently using ext4 for the filesystem. Maybe this is not the best choice from those available in Red Hat. Anyone have any recommendations that would speed up the process?

  2. The partition's mount options are sync,dirsync 1 2. Is there a way to optimize this since it's solely used for rsnapshot? Of course, reasoning would be greatly appreciated.

share|improve this question
1  
Perhaps noatime would be useful on the target. I doubt that would help with the rm though. I would think a non-journaling filesystem is what you're after. –  SpacemanSpiff Dec 25 '10 at 20:27
add comment

3 Answers 3

up vote 4 down vote accepted
  1. ext4 is fine.

  2. The "sync,dirsync" options make data and metadata updates synchronous, which is a major negative effect on performance on most workloads. Removing these options will very likely increase performance, but you have to remember to unmount the device before yanking the cable, otherwise you might lose data (presumably this is why the options were added in the first place, they are not the default options, or maybe it's some kind of special magic your distro does for USB devices).

  3. noatime disables atime updates, which reduces the amount of writes to the filesystem. More or less all applications, including rsnapshot, have no need for atimes, so this should be perfectly safe.

  4. data=writeback reduces the overhead of journaling, at a price of slightly increasing the possibility for lost data in case of a power failure. Depending on the distro this might already be the default.

  5. With ext4 it's also possible to disable the journal completely (as of kernel 2.6.29), although I wouldn't recommend that one. This still has all the other improvements of ext4, so this should be faster than using ext2, fwiw.

  6. barrier=0 disables barriers, which improves performance of writes, at the cost of increasing the possibility of data loss in a crash.

share|improve this answer
    
The cost of the noatime option for a few files is insignificant. It matters more for many, many files. –  Allen Dec 26 '10 at 15:23
    
noatime + async has brought down the process to a few minutes. Good call. –  Belmin Fernandez Dec 27 '10 at 1:12
add comment

You configure rsnapshot to keep more snapshots (for example 9999) and delete them yourself from disk using crontab. That will make speed of rnapshot more predictable.

share|improve this answer
add comment

Note also that using --link-dest also alters the plan for how the cp is done which can affect performance significantly (it basically just rms the hourly.$old and then uses rsync to do the copy from hourly.1 onto hourly.0 and sync from the source simultaneously).

There's more discussion on an alternative technique here - basically rotating the last hourly snapshot into the new one with mv hourly.$old hourly.0 instead of rm-ing it, and doing a cp -afl from hourly.1 onto hourly.0 to get it up to date but I haven't tried that - you'd have to do the rotation manually rather than relying on rsnapshot to do it

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.