Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have a big problem in understanding rsync filter rules for sub path. I have this directory tree:

|-- index.php
|-- other-file-php.php
|-- filesource.php
|   |-- album1
|   |   |-- firstphoto.jpg
|   |   |-- second.jpg
|   |   |-- index.php
|   |   `-- thumbs
|   |       |-- _map.txt
|   |       |-- _title_ciao.jpg.txt
|   |       |-- _120_second.jpg
|   |       |-- _dir_album1.jpg
|   |       `-- _300_second.jpg
|   |-- altre
|   |   |-- img_1172.jpg
|   |   |-- album2
|   |   |   |-- index.php
|   |   |   `-- thumbs
|   |   |       |-- _title_img_1172.jpg.txt
|   |   |       |-- _dir_album2.jpg
|   |   |       `-- _guestbook.html

I can only use option -f -filter of rsync, because I must write all rules into one file. In this tree there are different types of files that can be at any level of the tree. I want to:

  1. have a full backup of the directory structure
  2. backup all files of type .php, .txt, .html
  3. backup .jpg files that do not begin with _
  4. backup .jpg files that start with _dir
share|improve this question
add comment

3 Answers

up vote 2 down vote accepted

To address your points:

  1. This just means that all exclusion rules you specify must be files, and not directories. Since rsync doesn't distinguish on type, you've got to be careful with your rules (and file naming conventions).
  2. Use:

    + **/*.php
    + **/*.txt
    + **/*.html
    
  3. Use:

    + **/*.jpg
    - **/_*.jpg
    
  4. I have no idea what you mean.

share|improve this answer
add comment

Best for you is to make file you will use with option: --files-from using find command, like that:

cd dirtobackup;find . |egrep '\.php$|\.txt$|\.html$|^[^_].*\.jpg$|^_dir.*\.jpg$' >/tmp/files_tobackup
rsync --files-from=/tmp/files_tobackup dirtubackup dst
share|improve this answer
    
sorry but I can only use option -f -filter of rsync. and i have many many files... a find command would slow –  StefanoCudini Aug 5 '12 at 1:18
    
find will be same slow as rsync, as you still need to recursive crawl whole dir –  eicto Aug 5 '12 at 1:21
add comment

Put the following lines into filter.txt:

+ */
+ *.php
+ *.txt
+ *.html
+ **/_dir*.jpg
- **/_*.jpg
+ *.jpg
- *

Then run rsync like this:

rsync -a --include-from=filter.txt /path/to/source/ /path/to/dest

Explanation of filter.txt:

First of all, note that the ordering of the filter rules is very important, since rsync evaluates them in order and applies the first rule that matches.

By default, rsync includes all files that are not explicitly excluded, so the final rule in filter.txt is to exclude everything that wasn't explicitly included by the preceding rules. The first rule includes all directories (as indicated by the trailing /), which should take care of your #1 condition. Rules 2 thru 4 handle php, txt, and html files, respectively, while rules 5 thru 7 allow all jpg files that either start with _dir, or otherwise do not start with _.

(Rules 5 and 6 have the ** prefix in order to anchor the wildcard rules to the beginning of a filename part at any directory depth. Without the ** prefix, those rules would only match at the top level source directory.)

Final note: If you are unable (for some reason) to use --include-from, then you should be able to specify the filter rules as a series of -f options on the command line.

EDIT

To use filter.txt in rsnapshot, you can specify it in rsnapshot.conf via the include_file parameter, as follows:

include_file /path/to/filter.txt

EDIT #2

If you need to use a different set of filters for each backup, you can use a different include_file for each backup line in your rsnapshot.conf, e.g.:

...
backup /path/to/src1/ dest1/ include_file=/path/to/src1_filter.txt
backup /path/to/src2/ dest2/ include_file=/path/to/src2_filter.txt
...
share|improve this answer
    
tnk! but the problem is that I can only use option -f -filter of rsync. Because passing this option value by Rsnapshot –  StefanoCudini Aug 5 '12 at 4:38
    
Well then, check the rsnapshot man page. The include_file parameter in rsnapshot.conf allows you specify a file name that gets passed directly to rsync via --include-from. (See my edit.) –  Steven Monday Aug 5 '12 at 4:43
    
Actually, yes, you can use a different include_file for each backup rule. See my second edit. –  Steven Monday Aug 6 '12 at 1:42
    
:\ mmmh I'd edited –  StefanoCudini Aug 6 '12 at 1:50
    
However, before the path of filter is necessary "include_file=._/path/src.." –  StefanoCudini Aug 6 '12 at 1:52
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.