Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I'm wondering if its safe to use multiple rsync instances with the same command line to backup the same directory on the local server to the same directory on the remote server? basically if I ran the command below 3-4 times in parallel would it help speed things up? The recieving server is using rsyncd.

rsync -av /home/directory/ backups@1.1.1.1::Home

share|improve this question

3 Answers 3

up vote 3 down vote accepted

No. You can break it up into multiple commands, excluding or specifically including directories, but running the same command several times will result in a mess (especially with the rsync temp files).

share|improve this answer
1  
It also would not speed things up, unless both the source and destination systems were both on some kind of stripped volume with lots of drive spindles. On a system with one drive, the seeking all over the place. –  Zoredache Nov 18 '11 at 22:59

'Rsync' is already know for speeding up the transformation by transferring the selective and required part of that particular file or directory. Using the your mentioned command 3-4 times in parallel totally depends upon how much resources for your web hosting account has been granted, but one thing is certain that there would be more load created on server, so I would suggest you to do not go for it.

Rsync features

share|improve this answer

From version > 3.0.5 (maybe previous versions) you can use the --include, --exclude parameters.

Taken the example from here you could do something along these lines:

Assuming that directory_a has 2,000,000 files underneath it. and directory_b also has 2,000,000 files, running 2 separate rsync processes speed up the whole procedure in expense of CPU cycles.

rsync -av --include="/directory_a*" --exclude="/*" --progress remote::/ /localdir/ >  /tmp/myoutputa.log &
rsync -av --include="/directory_b*" --exclude="/*" --progress remote::/ /localdir/ >  /tmp/myoutputb.log &

According to the source this approach speeds up syncing by ~ 50%

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.