1

I'm using Ubuntu 12.04 with 2 x 2TB SATA3 harddisks in software RAID1. The filesystem is ext4 with data=writeback.

When using find . -maxdepth 1 -name '*' -delete to delete a large number of files in a directory, doing df at intervals show that:

Initially a large number of files get deleted, and iotop shows the find operation taking most of the IO.

enter image description here

Later df shows that the number of files remain constant, and iotop shows that IO usage is now dominated by jbd2 and md4_raid1.

enter image description here

Few minutes later, df shows the number of files dropping once more.

Can anyone explain this behavior? Is this normal or is there something wrong with my server? Also, how can 3 processes all have 99.99% IO usage at the same time?

2 Answers 2

2

It's what I would expect. Find finds files - i.e. it reads from the VFS, when the data is not in cache it needs to be fetched from disk. As it processes each file, it enqueues the changes to be written. At some point the write buffer gets full and the changes must be written to the disk - at this point the process doing the writes will be blocked making further write OR READ requests until there's space to buffer more writes.

3
  • When this mass deleting is done, the server slows to an almost complete halt. Is there a way for the server to avoid getting thrashed by the write block?
    – Nyxynyx
    Mar 14, 2013 at 13:43
  • Don't delete so many files all at once? Mar 14, 2013 at 13:50
  • See serverfault.com/questions/487599/…
    – symcbean
    Mar 14, 2013 at 14:25
0

You can prevent the system from being blocked by reducing the I/O priority of the process:

ionice -c3 nice -15 find ... -delete

If that's not enough then you can use chrt to put the process into the SCHED_IDLE class.

2
  • Should I actually reduce the IO priority of [jbd2/*] and [md4_raid1]? Will that slow down my entire IO subsystem even more?
    – Nyxynyx
    Mar 15, 2013 at 17:00
  • Don't touch that kernel stuff. I even doubt that it's possible to change their priority anyway. Mar 15, 2013 at 17:05

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .