Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Unfortunately we have a pretty bad HDD performance on one of our servers and as soon as we start bigger copy process it stops to deliver any Apache response, which is really bad.

Is there a way to limit the bandwidth of a copy process, so it doesn't consume all the HDD performance and still leaves some space for Apache and other process to do their job?

share|improve this question
    
Is there any option to correct the actual disk I/O issue? –  ewwhite Nov 11 '13 at 14:26
    
@ewwhite: unfortunately not in our budget range :) –  Tobias Zander Nov 11 '13 at 14:54
    
could also use "scp -l", but yeah as ewwhite pointed you have bigger issues than limiting local copy rate. –  Danila Ladner Nov 11 '13 at 15:09

4 Answers 4

up vote 4 down vote accepted

You could try rsync with the bandwidth limiter option set, e.g., copy at ~5MB/s

rsync -r /path/to/src /path/to/dest --bwlimit=5120 --progress

The --progress parameter is optional, but will confirm the copy bandwidth and progress. -r is for copying directories recursively rather than files.

share|improve this answer

Unfortunately I am not allowed to comment answers, so here is a small extension to the answer given by plasmid87: if you try to copy folders recursively, you should add -r parameter to the command. Besides that it works perfect.

share|improve this answer
    
+1 Good catch, thank you! –  plasmid87 Nov 12 '13 at 9:16

Your best bet, IMHO, is to run the cp process nice'd, reducing it's priority. It won't limit the bandwidth of the process per se, what that will do is to limit the amount of attention from the kernel / CPU the copy process gets relative to all the other processes on the system. With Apache at a normal priority, it should then be able to server normally een while the copy is going on.

share|improve this answer

It isn't clear if the copy processes you are running are part of the general server workload or something that can be scheduled for when there is little load on the machine.

If copying is done frequently and you are using a kernel > 2.6.25 you may find ionice of use.

From the man page:

This program sets or gets the io scheduling class and priority for a program. If no arguments or just -p is given, ionice will query the current io scheduling class and priority for that process.

For example, to copy a file using the 'idle' scheduling class, you can do something like:

ionice -c 3 cp /tmp/example.gpg /tmp/another_example.gpg

The -c 3 switch equates to selecting the idle scheduling class, about which the manual states:

A program running with idle io priority will only get disk time when no other
program has asked for disk io for a defined grace period. The impact of idle 
io processes on normal system activity should be zero.

This seems to me to be precisely what you are looking for.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.