I have a Xen Dom0 (3.2) with a bunch of domUs (paravirt). DomU VBDs are on local LVM volumes. Now I want to get backups of the LVM volumes using snapshots. I use rsync ioniced to idle (ionice -c3).

I have verified that the rsync processes are really in idle mode and the blkback kernel threads are on best efford (-c2 -n4).

As I understand the IO scheduling system, the so configured rsync process should never be able to steal IO bandwidth from the blkback processes and therefor the impact on the domUs should be zero.

But that's not what I see. In the domUs vmstat shows up to 20% IO wait during rsync filelist operation and about 10% on rsync transferring data:

r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
1  3    456  48468 103004 1550936    0    0    75   484 2112 1741  3  0 81 16

Without rsync running the IO wait is around 1%:

r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
1  0    456  43876 105564 1553224    0    0    19   365 2063 1718  4  0 95  1

dom0 shows steady 11% IO wait on rsync filelist operation and 5% on data transfer:

 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa
 0  0  23536 220220  73256 558856    0    0  1471   239 5271  310  0  0 89 11

So, has anyone an idea about what's happening here? Is ionice simply broken or am I expecting the wrong effects?

Thanks for any hint.



What distribution you have in use?

What file system you have in use?

What I/O scheduler (CFQ, deadline, anticipatory, noop) you have in use?

During heavy file operations 0% I/O wait is not a reasonable thing to expect. There WILL be some I/O wait with rotational disks, no matter what. Some file systems reduce that effect a bit, sometimes tuning or changing I/O scheduler can help a bit, sometimes you'll just have to live with a little bit of I/O wait. Around 10-20% is not THAT much unless you notice that as a dramatic slowdown with your server. In case of dramatic slowdowns you might get hint about what's going on with tools like btrace, which shows individual I/O requests, their status and overall I/O scheduler / block device statistics.

  • This is Ubuntu on CFQ Scheduler. It's not that I'm expecting 0% IO wait. I'm expecting nearly 0 impact from idle-niced IOs on "best-effort"-niced IOs. – tim Jul 27 '11 at 10:52
  • Sry, Kernel is: 2.6.24-24-xen #1 SMP Tue Jun 30 21:53:02 UTC 2009 x86_64 GNU/Linux – tim Jul 27 '11 at 11:25

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.