0

I have a script called copy-site which I use to copy a site from the production server to my local machine. copy-site example.net/httpdocs/ example.local/httpdocs/ will do a dry run rsync; adding a third parameter of true does the copy for real.

Sometimes, the local files have been edited. In this case, I want rsync to throw away my local edits, overwriting them with what's on the server. However, often it does not do this. Nor does it show anything relevant in an error log. I tried redirecting stderr to a log file, but it's usually empty or contains warnings about directories it cannot access on the remote server; it says nothing about not overwriting certain files. And certainly it can access those files: if I delete the local copies, and then run it again, it creates them fine, as a perfect copy of what they are on the server.

I added the -I flag today, but it made no discernible difference.

#!/bin/bash

if [ "$3" = true ]
then
    echo "real rsync";
    echo "deleting files";
    find $2 -type f -name ".*" -delete;
    echo "rsync running now";
    rsync -avziI --verbose --exclude-from /home/trig/copy-site-exclude -e 'ssh -p 2201' trig@server.example.net:/var/www/vhosts/$1 $2 --append-verify --delete;
    notify-send -i "$([ $? = 0 ] && echo terminal || echo error)" "copy site" "rsync complete";
    echo "setting permissions";
    chmod -R 0777 $2;
    notify-send -i terminal "copy site" "full permissions granted";
else
    echo "dry run rsync";
    rsync -avziI --verbose --exclude-from /home/trig/copy-site-exclude -e 'ssh -p 2201' trig@server.example.net:/var/www/vhosts/$1 $2 --append-verify --delete --dry-run;
    notify-send -i terminal "copy site" "dry run complete";
fi

copy-site-exclude is merely a list of image thumbnails (identified as beginning with dots), which I don't care to transfer as they can be regenerated locally.

custom/public/images/.*.jpg
custom/public/images/.*.png
custom/public/images/.*.gif
custom/public/images/.*.jpeg

What flags do I need to pass to rsync to make it discard all local changes, so that my local folder is a perfect reflection of the source folder on the server?

Local OS: Ubuntu.
Server OS: CentOS.

  • If you run copy-site example.net/httpdocs example.local/httpdocs then you will run rsync -avziI --verbose --exclude-from /home/trig/copy-site-exclude -e 'ssh -p 2201' trig@server.example.net:/var/www/vhosts/example.net/httpdocs example.local/httpdocs --append-verify --delete and end up with example.local/httpdocs/httpdocs/.[your files here]. Add trailing slashes to both arguments to get what I expect is the required behaviour (i.e. no double httpdocs directories). – wurtel Jan 14 at 8:48
  • @wurtel. Oops. But that's not the issue. The issue is that sometimes a file will be edited locally, and should be overwritten back to the way it was, but is not; or edited on the server, which should overwrite the local, but doesn't. Usually small text/PHP files, only ~6 lines long. – TRiG Jan 14 at 9:39
  • Try adding the -c (--checksum) argument: skip based on checksum, not mod-time & size – parkamark Jan 14 at 10:53
  • @parkamark. Running that now, and (as I should have expected) it takes far longer. Is it actually downloading each file to calculate the checksum? – TRiG Jan 14 at 11:05
  • It's looking at the file contents, not just the mod-time and size, yes. I think I encountered this same issue with my backups and adding this flag fixed things. – parkamark Jan 14 at 11:08
0

Well, simplifying the command (removing the -z and --append-verify flags) seems to have done the trick. However, I would still like to know what is actually happening here. From my reading, those flags shouldn’t have the effect of silently failing to overwrite (or to delete) some local files.

The line is now

rsync -avi --verbose --exclude-from /home/trig/copy-site-exclude -e 'ssh -p 2201' trig@server.example.net:/var/www/vhosts/$1 $2 --delete;

I’m not sure whether this is an answer (I’ve fixed my immediate problem) or a new question (why did this happen?). I’ll leave it here, anyway.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.