Rsync inconsistency between machines

Question

I've only just started trying to use rsync to do some backups from a couple of Linux machines to a Windows server, which runs the rest of the backup system. After a whole lot of reading and experimenting I finally got it all working on one of the Linux machines (Centos), so I copied the script over to the other machine (Debian), edited some paths and tried to run it. Instead of running the way it did on the first machine I get the following result:

building file list ...
1 file to consider

sent 36 bytes  received 16 bytes  11.56 bytes/sec
total size is 0  speedup is 0.00

The script in question is:

#! /bin/sh
/usr/bin/rsync \
--progress \
--recursive \
-t \
--delete-excluded \
--filter="- .cpan/***" \
--filter="- include/***" \
--filter="- cache/***" \
--filter="- lib/***" \
--filter="- cvs/***" \
--filter="+ /BayesTraining/***" \
--filter="+ /BogoTraining/***" \
--filter="+ /etc/***" \
--filter="+ /var/***" \
--filter="+ /root/***" \
--filter="+ /boot/***" \
--filter="+ /usr/local/***" \
--filter="+ /home/***" \
--filter="- *" \
/ \
rsync://myserver/targetfolder

It will only start sending files if I remove that last filter parameter but then it sends a whole bunch more than I want.

Where have I stuffed up and what do I need to change? (Before anyone asks, I'm not using rsync over SSH because the Windows server does not yet have an SSH service on it yet. Small steps...)

Edit: rsync --version reveals

Centos: rsync  version 2.6.8  protocol version 29
Debian: rsync  version 2.6.4  protocol version 29

Solution: Thanks to pavium for pointing out that I have overlooked a crucial part in the man page (reformatted for readability):

a trailing "dir_name/***" will match both the directory (as  if "dir_name/"
had been specified) and everything in the directory (as if "dir_name/**"
had been  specified). This  behavior  was added in version 2.6.7.

The solution is to create the following rules for each directory I want included:

--filter="+ /dirname/"
--filter="+ /direname/**"

Answer 1

The last filter item may be your problem (well, I think you know that). I found the following in the rsync 2.6.9 man page:

Note that, when using the --recursive (-r) option (which is implied by -a), every subcomponent of every path is visited from the top down, so include/exclude patterns get applied recursively to each subcomponent’s full name (e.g. to include “/foo/bar/baz” the subcomponents “/foo” and “/foo/bar” must not be excluded). The exclude patterns actually short-circuit the directory traversal stage when rsync finds the files to send. If a pattern excludes a particular parent directory, it can render a deeper include pattern ineffectual because rsync did not descend through that excluded section of the hierarchy. This is particularly important when using a trailing ’*’ rule. For instance, this won’t work:

+ /some/path/this-file-will-not-be-found
+ /file-is-included
- *

This fails because the parent directory “some” is excluded by the ’*’ rule, so rsync never visits any of the files in the “some” or “some/path” directories. One solution is to ask for all directories in the hierarchy to be included by using a single rule: “+ */” (put it somewhere before the “- *” rule), and perhaps use the --prune-empty-dirs option. Another solution is to add specific include rules for all the parent dirs that need to be visited. For instance, this set of rules works fine:

+ /some/
+ /some/path/
+ /some/path/this-file-is-found
+ /file-also-included
- *

So, the - * is excluding paths not explicitly included earlier in the filter rules. The man page also tells me that the trailing "dir_name/***" behaviour (which would have enabled those paths) was added at version 2.6.7 (your Debian is earlier)

Answer 2

Psychic debugging mode activated:

Bash is probably expanding the * into a list of all files in the current directory, so rsync doesn't see a * it sees file1 file2 file3.

Try escaping the * - --filter="- \*" \, or --filter='- *' \ should do the trick.

Answer 3

Run 'rsync --version' on your CentOS and Debian boxes, and compare the results. Is the Debian one older perhaps? It's possible it's a fault of that particular version or the way it was compiled on that box.