13

I'm currently using rsync on a script that deploys a PHP application from a staging to a production server. Here is how:

rsync -rzai --progress --stats --ignore-times --checksum /tmp/app_export/ root@app.com:/var/www/html/app/

This is currently outputting a list of every file that's being compared (every file in the project), but I'd like it to output only the modified ones, so I can run it with a --dry-run option to check that every deploy is updating only the desired files.

NOTE: The best I could do so far is grep fcst the results, but I'm looking for an rsync option that I'm sure it's there but I can't find it in the man pages.

Thanks in advance!

5

If there's an rsync option to do exactly what you're asking, I didn't find it in the manpage either. :-)

That said, I don't see the problem with grepping the output of rsync -i to parse out exactly what you need. That feels nice and Unixy to me.

One nit-picky quibble with your rsync command: the -r is redundant, as it is implied by -a.

  • 2
    Thanks man. I wonder why (with the variety of options this program has) there isn't an option for that. Seems pretty basic for me. – Mauro Sep 11 '12 at 22:15
  • Hi Kevin, actually there is an option: -P for progress does exactly that in the -n dryrun setting, although -i isn't wrong either. See my answer below. – Robert Riedl Jan 31 at 8:20
5

Use the --out-format option

According to the man page:

Specifying the --out-format option will mention each file, dir, etc. that gets updated in a significant way (a transferred file, a recreated symlink/device, or a directory).

If you only need the actual filenames (--out-format="%n") your dry run command might look like:

rsync -rzan --out-format="%n" --ignore-times --checksum /tmp/app_export/ root@app.com:/var/www/html/app/


When rsync is called with -v, it internally uses this option with a default format of "%n%L", which tells you just the name of the file and, if item the is a link, where it points.

But this also includes a short summary at the beginning and the end of the sync process.

To get rid of that summary use the --out-format option directly.

Btw. -i also internally uses --out-format, but with a format of "%i %n%L".

3

Beginning with rsync v3.1.0, released in 2013, there is the --info flag which allows fine-grained control over the output.

 --info=FLAGS
          This option lets you have fine-grained control over the information output you want to see.  An individual flag name may be followed
          by a level number, with 0 meaning to silence that output, 1 being the default output level, and higher numbers increasing the output
          of that flag (for those that support higher levels).  Use --info=help to see all the available flag names,  what  they  output,  and
          what flag names are added for each increase in the verbose level.  Some examples:

              rsync -a --info=progress2 src/ dest/
              rsync -avv --info=stats2,misc1,flist0 src/ dest/

          Note  that  --info=name’s  output  is  affected  by the --out-format and --itemize-changes (-i) options.  See those options for more
          information on what is output and when.

          This option was added to 3.1.0, so an older rsync on the server side might reject your attempts at fine-grained control (if  one  or
          more  flags  needed  to  be  send to the server and the server was too old to understand them).  See also the "max verbosity" caveat
          above when dealing with a daemon.

The available --info flags are:

Use OPT or OPT1 for level 1 output, OPT2 for level 2, etc.; OPT0 silences.

BACKUP     Mention files backed up
COPY       Mention files copied locally on the receiving side
DEL        Mention deletions on the receiving side
FLIST      Mention file-list receiving/sending (levels 1-2)
MISC       Mention miscellaneous information (levels 1-2)
MOUNT      Mention mounts that were found or skipped
NAME       Mention 1) updated file/dir names, 2) unchanged names
PROGRESS   Mention 1) per-file progress or 2) total transfer progress
REMOVE     Mention files removed on the sending side
SKIP       Mention files that are skipped due to options used
STATS      Mention statistics at end of run (levels 1-3)
SYMSAFE    Mention symlinks that are unsafe

ALL        Set all --info options (e.g. all4)
NONE       Silence all --info options (same as all0)
HELP       Output this help message

Options added for each increase in verbose level:
1) COPY,DEL,FLIST,MISC,NAME,STATS,SYMSAFE
2) BACKUP,MISC2,MOUNT,NAME2,REMOVE,SKIP
  • This is awesome, thanks. I had no idea this flag existed because I use a Mac - even the latest MacOS installs an rsync version from 2004 lol. Cheers Apple... – twistedpixel Feb 5 '18 at 15:00
0

Not sure if this differs between versions/options used but in mys version when I use -i option I get a list like:

>f..T...... existing-file.png
>f+++++++++ new-file.png
cd+++++++++ new-dir/
>f+++++++++ new-dir/new-file.png

So a simple solution to get only a list of files actually transfered just run:

rsync [your options here] | grep -v "f..T......"

This will simply hide all lines containing f..T....... So effectively this will hide identical files.

0

Actually, you kind of answered your own question, since your original command already has it: --progress

This is the correct option, although the man page is a bit cryptic about it:

     --progress              show progress during transfer
 -P                          same as --partial --progress

It kinda makes sense, since you invoke your rsync string with dryrun mode, no transer happens, but you'd still have progress: namely the files that have changed and would be transferred.

This way you get a neat little list of all the files, for example:

The destination already has a copy of changedfile which was updated in the source and oldfile, which remains unchanged. The source has also an additonal file: newfile.

#~$ ls -lhan /tmp/destination/
total 20K
drwxrwxr-x  2 1000 1000 4,0K Jän 31 09:07 .
drwxrwxrwt 18    0    0  12K Jän 31 09:15 ..
-rw-rw-r--  1 1000 1000    2 Jän 31 09:08 changedfile
-rw-rw-r--  1 1000 1000    0 Jän 31 09:07 oldfile



#~$ ls -lhan /tmp/source/
total 20K
drwxrwxr-x  2 1000 1000 4,0K Jän 31 09:07 .
drwxrwxrwt 18    0    0  12K Jän 31 09:15 ..
-rw-rw-r--  1 1000 1000    2 Jän 31 09:15 changedfile
-rw-rw-r--  1 1000 1000    0 Jän 31 09:07 newfile
-rw-rw-r--  1 1000 1000    0 Jän 31 09:07 oldfile

If we then invoke your rsync command, but remove itemization -i and just add dryrun -n

#~$ ~$ rsync -n -rza --progress --stats --ignore-times --checksum /tmp/source/ /tmp/destination/
sending incremental file list
changedfile
newfile

Number of files: 4 (reg: 3, dir: 1)
Number of created files: 1 (reg: 1)
Number of deleted files: 0
Number of regular files transferred: 2
Total file size: 2 bytes
Total transferred file size: 2 bytes
Literal data: 0 bytes
Matched data: 0 bytes
File list size: 0
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 187
Total bytes received: 22

sent 187 bytes  received 22 bytes  418.00 bytes/sec
total size is 2  speedup is 0.01 (DRY RUN)

You get a list of only the files rsync would tranfser: changedfile and newfile.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.