44

I am using rsync in a bash script to keep files in sync between a few servers and a NAS. One issue I have run into is trying to generate a list of the files that have changed from the during the rsync.

The idea is that when I run rsync, I can output the files that have changed into a text file - more hoping for an array in memory - then before the script exists I can run a chown on only the changed files.

Has anyone found a way to perform such a task?

# specify the source directory
source_directory=/Users/jason/Desktop/source

# specify the destination directory
# DO NOT ADD THE SAME DIRECTORY NAME AS RSYNC WILL CREATE IT FOR YOU
destination_directory=/Users/jason/Desktop/destination

# run the rsync command
rsync -avz $source_directory $destination_directory

# grab the changed items and save to an array or temp file?

# loop through and chown each changed file
for changed_item in "${changed_items[@]}"
do
        # chown the file owner and notify the user
        chown -R user:usergroup; echo '!! changed the user and group for:' $changed_item
done
1
  • → see my anwer here. Also -i for itemize, but with a few more twists...
    – Frank N
    Feb 26, 2017 at 12:45

4 Answers 4

66

You can use rsync's --itemize-changes (-i) option to generate a parsable output that looks like this:

~ $ rsync src/ dest/ -ai
.d..t.... ./
>f+++++++ newfile
>f..t.... oldfile

~ $ echo 'new stuff' > src/newfile

~ $ !rsync
rsync src/ dest/ -ai
>f.st.... newfile

The > character in the first position indicates a file was updated, the remaining characters indicate why, for example here s and t indicate that the file size and timestamp changed.

A quick and dirty way to get the file list might be:

rsync -ai src/ dest/ | egrep '^>'

Obviously more advanced parsing could produce cleaner output :-)

I came across this great link while trying to find out when --itemize-changes was introduced, very useful:

http://andreafrancia.it/2010/03/understanding-the-output-of-rsync-itemize-changes.html (archived link)

3
  • 3
    For a cleaner output as mentioned, rsync -zaic src/ dest/ | grep '^?c' | cut -d' ' -f2 --dry-run would only list the modified files (different checksum) , definitely a keeper, thx :) FYI putting --dry-run after the command instead of using the n option is for me a best practice
    – MediaVince
    Dec 23, 2016 at 9:37
  • Essentially the same as rsync -zavc src/ dest/ --dry-run without verbose
    – MediaVince
    Dec 23, 2016 at 9:45
  • 2
    Awesome, I didn't know the ! syntax for using previous parameters... Thanks a lot!
    – Niloct
    Oct 13, 2020 at 2:42
20

Use the -n flag, combined with the -c checksum flag and the -i flag:

# rsync -naic test/ test-clone/
>fcst...... a.txt
>fcst...... abcde.txt
>fcst...... b.txt

In this example, three files have changed, based on the contents (as determined by the checksum) of the file itself. However, no file syncing is done because of the -n flag

Bonus

If you want to run chown on the changed files, parse them out with sed or similar and process with xargs, for example:

rsync -naic test/ test-clone/ | sed 's/............//' | xargs -I+ sudo chown root "test-clone/+"
5
  • 4
    sed -> cut -d ' ' -f2,- Jun 21, 2016 at 12:01
  • 4
    -n --dry-run, -c --checksum, -i --itemize-changes Aug 18, 2018 at 20:12
  • 2
    I'm not following. All 3 of those files in the example are marked '>fcst' which I understand means 'being received from the remote', checksum differs, size differs, time differs. What in the output indicates that "only one file has changed"??
    – bobpaul
    Nov 1, 2019 at 23:14
  • @bobpaul: I was puzzling over the same question when I read your comment (great minds...? :)
    – Seamus
    Mar 25, 2020 at 11:22
  • thanks for the clarification, @Seamus
    – JDS
    Oct 2, 2020 at 17:59
9

This question is a little bit old, but I think it worth to be added:

-i is a shortcut of --out-format='%i %n%L'

%i represents a cryptic, 11 character output whose format is the string YXcstpoguax; ref man rsync, -itemize-changes. And %n means the filename, (section log format of man rsyncd.conf); %L comes into play when there are symlinks to update.

P.S. rsync version 3.1.0

1
1

Summarizing a few other answers (especially @Cychih's), you can get the list of changed files like so:

rsync --out-format='%n' src/ dest/

Which will print only the changed files, eg;

rsync --out-format='%n' src/ dest/
a.txt
bcde.txt
b.txt

You can save that to a list this way:

changed_items=($(rsync --out-format='%n' src/ dest/))
for item in "${items[@]}"; do
   echo $item
   echo $item
done

You can pipe them to another command like so:

rsync --out-format='%n' src/ dest/ | xargs open

Note that it's very common to include -acz (archive, checksum, and compress) flags as well.

2
  • 3
    rsync -r --out-format='%n' src/ dest/ seemed to print every file for me.
    – Kingsley
    Nov 24, 2021 at 1:38
  • Same for me, this just prints every file, just the same as -v, but excluding the summary at the end Nov 14, 2023 at 16:45

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .