I would like to backup 1 TB on a weekly basis from PROD to BACKUP server. The servers run under Linux Ubuntu-1004-lucid-64-minimal 2.6.32-35-server.
There is the BACKUP server (where I start the rsync program) and the PROD server with the data.
The command I currently use is the following:
time rsync -r --delete firstname.lastname@example.org:/home/myuser/data .
The issue with this is: the PROD server becomes nearly unresponsive. The web application that runs on the PROD server is nearly dying.
UPDATE: Current working solution
After some feedback, I'm using this command now for doing the backup of 1TB of data and it is absolutely working fine:
rsync -r --delete --rsync-path "ionice -c 3 nice rsync" --bwlimit=30000 email@example.com:/home/myuser/data .
Please note that I have set a bandwidth limit to 30 Mbps because my connection between PROD and backup server has 100 Mbps and that bandwidth is shared with the production traffic of my web applications.
Please note that I execute this command on the BACKUP server, so that's why I'm using the rsync-path option in order to nice and ionice the process on the remote server (PROD).
My Original questions for possible Solutions
How can I control the impact of rsync (started on the BACKUP server) ?
How would you solve this issue?
My little research brought out the following possibilites:
Execute rsync in a way so that the 1TB is synched in chunks? e.g.
rsync /source/[0-9]* firstname.lastname@example.org:/source_backup rsync /source/[a-h]* email@example.com:/source_backup/ rsync /source/[i-p]* firstname.lastname@example.org:/source_backup/ rsync /source/[q-z]* email@example.com:/source_backup/
Would it help to limit bandwidth with the option
Is it possible to nice the process on the remote machine in a way? e.g.
nice -n19 backup.sh
I don't know whether the process on the PROD machine would be niced too?
Any help and ideas is very welcome.