ionice man page says
A program running with idle io priority will only get disk time when no other program has asked for disk io for a defined grace period.
Where is this "grace period" defined? Is it visable/tunable (perhaps via /sys)?
The grace period is 250 milliseconds.
There is an idle scheduling class, which only runs when nothing else
is using the disk. A grace period is defined for which idle has to
wait before getting disk access when other io has run. This defaults
to 250ms currently.
You didn't provide the OS version or distribution (and yes, it does make a difference), but the tunable for RHEL/CentOS 5+6 is defined here as fifo_expire_async in
/sys/block/[device]/queue/iosched/ (where [device] is sda, sdb, cciss0, etc.).