Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

My tar command

tar --remove-files -cvif 2011-08-02_14-05-09.tar 2011-08-02_14-05-09

Writes 80MB/s. That's beyond the capacity we can spare because MySQL starts queuing up writes and eventually we get monitoring alerts cause systems won't wait forever on MySQL and time out.

Q: Can I throttle tar to only do 20MB/s?

Didn't find anything in the manpage and we already tried with the lowest ionice class: /usr/bin/ionice -c3 tar <...> but that doesn't impact the MBs written/s and MySQL still freezes up.

Maybe ionice is ignored because we're on a /dev/md0 software RAID device?

share|improve this question
    
I guess as a hack we could immediately let tar bzip the archive cause we do have CPU cycles to spare. But I'm really interested to see if I can cap IO in a solid manner. –  kvz Aug 2 '11 at 14:39
    
ionice not working for you may have been because you were running with the incorrect scheduler. It apparently only works with the CFQ scheduler. –  Stu Thompson Oct 31 '11 at 12:28
add comment

2 Answers 2

up vote 4 down vote accepted

You can use ratepipe.

I downloaded and tried it out and it seems to DTRT. It should work for you:

tar --remove-files -cvif - 2011-08-02_14-05-09 | ratepipe -r 20 > 2011-08-02_14-05-09.tar
share|improve this answer
1  
Great idea! I found an alternative called pv (linux.die.net/man/1/pv) that can be found in apt. Syntax would become tar --remove-files -cvif - 2011-08-02_14-05-09 | pv -L 20m > 2011-08-02_14-05-09.tar Can also e.g. show a progress bar –  kvz Aug 2 '11 at 15:32
1  
That's it! I knew there was a sexier program around, but couldn't remember pv. –  MikeyB Aug 2 '11 at 16:09
    
This is a gem of a find. Thanks. ratepipe rocks. –  Stu Thompson Oct 31 '11 at 10:37
add comment

You could probably write the tar to stdout (using -f -) and pipe it into something that throttles. I'll see if I can find something quickly.

Since tar will not buffer much, it should wait on the reads.

Edit: MikeB beat me to it: ratepipe will do.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.