30

When using rsync+ssh to access a remote machine, is there a way to "nice" the rsync process on the remote machine (to lower its priority)?

Editing the question to clarify:

USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
backups  16651 86.2  0.1   3576  1636 ?        Rs   11:06   0:06 rsync --ser...

(rsync line snipped)

This is a backup cron job that normally runs at 4am, but when I happen to be awake (and committing, or using Bugzilla hosted on that same machine), it kills server performance, so I wanted a quick "hack" to try and fix it a bit.

Improve this question

5 Answers 5

Reset to default
38

You can use the --rsync-path option, eg.

rsync --rsync-path="nice rsync" foo remotebox:/tmp/
Improve this answer
4
  • 23
    You might consider using ionice as well, e.g. --rsync-path="ionice -c 3 nice rsync", though modern Linuxes automatically reduce the IO priority for nice'd processes (see the man page I linked).
    – Jo Liss
    Jun 23, 2011 at 12:57
  • I already tried that, but my sshd is still running at nice 0
    – Felipe Alvarez
    Dec 3, 2014 at 5:09
  • on FreeBSD like systems where you don't have ionice, setting the --bwlimit option is good for a nicer rsync.
    – deed02392
    Jun 22, 2016 at 8:00
  • 3
    I also had a problem with this option not making any difference. It turned out that the server I was connecting to was configured in authorized_keys to only accept a specific rsync command, which was overriding the --rsync-path option I was specifying.
    – Andy Beverley
    Oct 25, 2016 at 10:57
13
  • Taking the --rsync-path option you have rsync --rsync-path="ionice -c 3 nice -n 12 rsync" localDirectory remoteHost:/tmp/
  • Taking the configuration file option you can change or uncomment in file /etc/default/rsync the RSYNC_NICE='17' value and the RSYNC_IONICE='-c3' value

For both the ionice value will be for hard disc priority

  • 1 -> Real time
  • 2 -> Best effort
  • 3 -> Ildle (when any other process is not using the HD)

Note that for Linux only cfq scheduler really implements IO classes and priorities. If the remote system administrator has opted to IO scheduler noop, deadline or some more exotic variant you may find that ionice really does nothing. One should be able to get high performance from cfq and still be able to use IO classes and priorities by tuning cfq correctly.

for nice value for processor priority

  • -20 (most favorable to the process)
  • (default 10) if -n is not specified
  • 19 (least favorable to the process)
Improve this answer
2
  • Who is responsible for /etc/default/rsync? Is rsync itself reading this file, or is this done by the linux-flavor/distribution?
    – guettli
    Apr 20, 2016 at 9:24
  • ... I think I found the solution /etc/default/rsync is the defaults file for rsync daemon mode. I guess it only applies if you connect to the rsync daemon via rsync-protocol. I guess it does not apply if you call it via ssh.
    – guettli
    Apr 20, 2016 at 9:26
4

You could disable the compression along the network, by not including the -z argument, that might save some CPU time on either side. Or change how rsync uses checksums, look at --checksum

Improve this answer
2
  • I accepted Hasturkun's answer (since it answered my question), but you brought up a good point: I benchmarked with compression vs without compression, and it only added 2 minutes to the job. I just got so used to using -avz as my flags that I never thought of dropping -z.
    – mwalling
    Aug 18, 2009 at 15:35
  • My habit is to use -aPh, I rarely use -z
    – Amandasaurus
    Aug 18, 2009 at 15:57
2

Quick and dirty solution would be to create a small wrapper script called 'rsync' that shadows the $PATH before real rsync binary like:

#!/bin/sh
nice -10 /path/to/proper/rsync $*

Or setup the authorized_keys file so that it performs nicing of rsync. (Assuming you are using ssh keys).

example:

command=”/home/user/bin/nice-rsync.sh" ssh-dss asdf....

Now in your /home/user/bin/nice-rsync.sh

#!/bin/sh
case $SSH_ORIGINAL_COMMAND in
  rsync\ --server*)
    nice -10 $SSH_ORIGINAL_COMMAND
    ;;
  *)
    $SSH_ORIGINAL_COMMAND
    ;;
esac

HTH

Improve this answer
0

Rsync should not be using much CPU. I doubt that you can force a certain niceness from the other end, but what you could do is limit the bandwidth that rsync is using with a firewall, which would ultimately reduce how much processing it could do in X amount of time.

Improve this answer
3
  • Added some output from 'ps aux' to show what it is doing as far as CPU. I'm going over a slow link, so I have the --compress option turned on, but it is also causing a good amount of disk access, which I was hoping to knock down by using nice (unintended side effect)
    – mwalling
    Aug 18, 2009 at 15:11
  • Setting a nice value usually also affects the I/O niceness, which is what someone usually wants.
    – Bram Schoenmakers
    Aug 24, 2010 at 7:46
  • rsync does not require much CPU but it might show up in top nevertheless depending if the current kernel considers I/O wait as busy. In addition, if any other process needs to access the storage and rsync is allowed to run at full pace all the other processes will be slowed down as a result.
    – Mikko Rantalainen
    Jun 24, 2019 at 6:01

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .