Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.
for f in `ssh $SSHCRED "~/file-list.sh"`
do
  rsync --progress --times --partial --append --rsh=ssh -r -h --remove-sent-files $SSHCRED:$f $OUTDIR
done

This is what I do today.

I'd like to instead do this by sending the output from ssh $SSHCRED "~/file-list.sh" directly to rsync. That should be faster, and I can easily abort just one rsync operation.

But I see at least one problem with this: The remote script generates a list of files with absolute paths. For rsync to work, I think I need to prepend the SSH credentials before every path, like this:

user@host:/path/to/file1
user@host:/path/to/file2
user@host:/path/to/file3

Is this possible?

share|improve this question
add comment

1 Answer

up vote 5 down vote accepted

Generate a list of files and hand them over to rsync with the --files-from=FILE option:

rsync --your-options --files-from=filelist.txt user@host:/basedir targetdir

This should be much faster than calling rsync for every single file.

share|improve this answer
    
Thanks! It almost works the way I want. The only problem is that it recreates the whole dir structure. So say I want to rsync every file in /home/backups to /var/backups on my local machine, it will create /var/backups/home/backups. Can I avoid this? –  Znarkus Dec 2 '11 at 14:14
    
Yes. Make sure the list has only relative filenames (subdir1/filename1 instead of /home/backups/subdir1/filename1 and then use user@host:/home/backups as from-target for rsync. –  SvW Dec 2 '11 at 15:31
    
Oh okay. That makes it more complicated, if I want the remote script to handle what files should be copied from where. Even if it's just from one folder. Maybe I could extract the folder path from the first line, then remove that string from every line. –  Znarkus Dec 3 '11 at 10:11
    
Or after the script has run, I could flatten the directory structure created by rsync, and leave a single folder with all files. –  Znarkus Dec 3 '11 at 10:13
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.