We're using rsnapshot for backups. It keeps lots of snapshots of the backuped up file, but it does delete old ones. This is good. However it's taking about 7 hours to do a rm -rf on a massive directory tree. The filesystem is XFS. I'm not sure how many files are there, but it numbers in the millions probably.

Is there anyway to speed it up? Is there any command that does the same as rm -rf and doesn't take hours and hours?

  • 1
    I used find . -delete -name directory and it's much more fast than rm -rf. – Paolo Dec 14 '13 at 1:13
up vote 38 down vote accepted

No.

rm -rf does a recursive depth-first traversal of your filesystem, calling unlink() on every file. The two operations that cause the process to go slowly are opendir()/readdir() and unlink(). opendir() and readdir() are dependent on the number of files in the directory. unlink() is dependent on the size of the file being deleted. The only way to make this go quicker is to either reduce the size and numbers of files (which I suspect is not likely) or change the filesystem to one with better characteristics for those operations. I believe that XFS is good for unlink() on large file, but isn't so good for large directory structures. You might find that ext3+dirindex or reiserfs is quicker. I'm not sure how well JFS fares, but I'm sure there are plenty of benchmarks of different file system performance.

Edit: It seems that XFS is terrible at deleting trees, so definitely change your filesystem.

  • 1
    Some years ago I noticed terrible performance using reiserfs in a similar use case. – knweiss Jul 28 '09 at 8:54
  • 1
    Marvelous post! – wzzrd Jul 28 '09 at 9:04
  • 2
    It nearly just said "no" :) – David Pashley Jul 28 '09 at 11:39
  • 2
    I agree with everything here apart from your statement that unlink speed being dependant on the size of the file. unlink just removes the link to the file and does nothing to the actual contents. There should be no discernable difference between files of different size (you can test this yourself). – Kamil Kisiel Jan 28 '10 at 22:27
  • @KamilKisiel You are right telling unlink does nothing to the actual content but to perform an unlink system call, the file system code has nevertheless more work to do if the removed link is the last one to the file and if it is not currently open. This is of course file system dependent but there can then be a very discernible difference when the removed file is huge. – jlliagre Sep 5 '16 at 19:56

As an alternative, move the directory aside, recreate it with the same name, permissions and ownership and restart any apps/services that care about that directory.

You can then "nice rm" the original directory in the background without having to worry about an extended outage.

  • That could work, since a mv is very very quick. – Rory Jul 28 '09 at 10:15
  • Yup - it works well. I've used this technique many times to "fix" maildir-based mailboxes where an email client has lost it brain and left a mess on the disk. The biggest (single) directory I've fixed in this manner had around around 1.5 or 2 million files IIRC. Total downtime to the end user was ~3 minutes, most of which was waiting for the mail client and imap processes to die. – Greg Work Jul 28 '09 at 11:25

Make sure you have the right mount options set for XFS.

Using -ologbufs=8,logbsize=256k with XFS will probably triple your delete performance.

  • 2
    +1 for this tip... One should also enable lazy counters for another performance boost. – hurikhan77 Sep 21 '09 at 21:36
  • 1
    Some explanation on these settings would be helpful for future readers. – Aron Rotteveel Oct 30 '15 at 11:35

If you are doing the rm at effectively at the file level then it will take a long time. This is why block based snapshots are so good:).

You could try splitting the rm into separate areas and trying to do it in parallel however I might not expect it to make any improvement. XFS is known to have issues deleting files and if that is a large part of what you do then maybe a different file system for that would be an idea.

  • Block-based snapshots are not uniquely good in this case. A number of file systems---WAFL and ZFS come immediately to mind---also provide good performance for snapshot delete. They treat snapshots as first class file system objects. So rather than iterating (slowly) over millions of files to determine which blocks to free, they only have to look a the block-list associated with the snapshot. – Keith Smith Jul 30 '09 at 1:03
  • Hmm. I probably came off as being too contrary above. The original poster must be using Linux, and there really isn't a well-proven Linux file system that does snapshots---although btrfs and nilfs look interesting for the future. So as a practical matter, I agree---better to use block-based snapshots. – Keith Smith Jul 30 '09 at 1:30
  • +1 for the tip to split and parallelize the workload: xfs plays its strength on parallel workloads. – hurikhan77 Sep 21 '09 at 21:38

It's good to use ionice for IO-intensive operations like that regardless of filesystem used.
I suggest this command:

ionice -n7 nice rm -fr dir_name

It will play nicely for background operations on server with heavy IO load.

I know this is old, but I thought id toss in a suggestion. You are deleting those files sequentially, executing parallel rm operations might speed things up.

http://savannah.nongnu.org/projects/parallel/ parallel can commonly be used in place of xargs

so if your deleting all the files in deltedir

find -t f deletedir | parallel -j 10 rm

That would leave you with just empty directory structures to delete.

Note: You will likely still hit the filesystem limitations as noted above.

  • What's advantage of using parallel over xargs? – Rory Jan 31 '10 at 22:48

Would an alternative option here be to seperate the data in such a way that you can junk and rebuild the actual filesystem instead of doing the rm?

  • 3
    I think rsnapshot uses hard-links as part of the maintaining-multiple-snapshots-efficiently feature. So if the questioner is using that feature using separate filesystems won't work (as you can't hard-link over a filesystem boundary) – David Spillett Jul 28 '09 at 10:10

How about decreasing the niceness of the command? Like:

nice -20 rm -rf /path/to/dir/
  • 5
    The bottleneck is not the scheduler, it is the file system, I would say. – Manuel Faux Jul 28 '09 at 8:41
  • In the unlikely event that the scheduler is the bottleneck, you'd only end up pounding the I/O subsystem harder, making the server even less usable during the rm. – David Mackintosh Jul 28 '09 at 15:44

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.