4

Trying to safely retrieve files from heavily infected Windows XP and 2003 machines that came to us from an acquisition. I have a similar need for automating a script that would run on PXE booted DRBL machines to rsync to our server.

The following code results in single quotes wrapping the entire '--filter=''' parameter to rsync. Note echo strips the single quotes, but not rsync. What am i doing wrong? Would making every parameter an entry in an array help?


FAILS because wrapping single quotes:

'--filter='\''- "Default User/"'\''  --filter='\''- "Application Data/"'\'' '


When i run the same exact command but manually remove the wrapping single quotes, rsync runs fine.
WORKS:

--filter='\''- "Default User/"'\''  --filter='\''- "Application Data/"'\'' 

Have tried without and without --protect-args.


#!/bin/sh -x

PROTECTARGS=--protect-args
EXCLUDE=( '- "Default User/"' '- "Application Data/"' )
echo EXCLUDE="${EXCLUDE[*]}" 
FILTER=()
for X in "${EXCLUDE[@]}"; do
  FILTER=( "${FILTER[@]}" "--filter='${X}' " );
done;
echo FILTER="${FILTER[*]}"

if [ "1" == "1" ]; then
rsync --dry-run -vv --stats \
      --recursive \
      "$PROTECTARGS"  \
      "${FILTER[*]}"  \
      '/hddOld/Documents and Settings' \
      '/hddNew/'
fi;

OUTPUT:


FILTER=--filter='- "Default User/"'  --filter='- "Application Data/"' 
+ '[' 1 == 1 ']'
+ rsync --dry-run -vv --stats --recursive --protect-args '--filter='\''- "Default User/"'\''  --filter='\''- "Application Data/"'\'' ' '/hddOld/Documents and Settings' /hddNew/
Unknown filter rule: `'- "Default User/"'  --filter='- "Application Data/"' '
rsync error: syntax or usage error (code 1) at exclude.c(817) [client=3.0.6]




bash-4.1.2-14.el6.x86_64
rsync  version 3.0.6  protocol version 30
rsync-3.0.6-9.el6.x86_64
CentOS 6.4 x64

WORKS ... NO IT DOES NOT:


FILTER+=( --filter=""${X}""  );

RESULTS IN:
'--filter=- "Default User/" --filter=- "Application Data/"'  

rsync does not complain about this filter rule but does not actually filter out "Application Data/".

2
  • A related question involving mutt, but the newer array syntax seems to solve it? serverfault.com/questions/420374/…
    – rjt
    May 7, 2013 at 21:37
  • No, the newer syntax does not solve it.
    – rjt
    May 7, 2013 at 22:10

2 Answers 2

1

I got it to work by doing this:

I changed:

EXCLUDE=( '- "Default User/"' '- "Application Data/"' )

to

EXCLUDE=( '-_Default User/' '-_Application Data/' )

This removed the double quotes and used the alternate underscore "_" syntax after the "-". A prior answer suggested this but I had to add the "_" syntax to get it to work for me.

Also changed:

FILTER=( "${FILTER[@]}" "--filter='${X}' " );

to

FILTER=( "${FILTER[@]}" "--filter=${X}" );

This removed the apostrophes from around the X variable substitution.

And changed:

"${FILTER[*]}"

to

"${FILTER[@]}"

This causes each element of the array to be treated as a single parameter, rather than the whole array as noted previously.

11
  • I was able to get it to work by just removing the single-quotes from the line which adds the component to the filter line; this is the second change you suggested.
    – Kevin M
    May 8, 2013 at 8:40
  • @KevinM, I tried that first, but rsync did not filter out "Default User" and "Application Data" (using a set of test directories). It wasn't until I made the other changes that it finally skipped those directories and picked up everything else under "Documents and Settings".
    – jay
    May 9, 2013 at 3:14
  • i am not sure why salvaging user profile data from a Win2003 server and 20 workstations is considered "off-topic". Professionals need to know how to use rsync and bash scripting. A question gets summarily closed with no warning.
    – rjt
    May 9, 2013 at 16:52
  • i tried the suggestions above and non worked. It may be possible that that the posting process removed some relevant quoting characters. Please double check what is on the webpage is exactly the same as in your script. Please review and try this test script: bitbucket.org/rjt69/scratchpad/src/bbcaa52e8cd1/…
    – rjt
    May 9, 2013 at 17:00
  • Instead of closing questions, they could be tagged with another StackExchange website.
    – rjt
    May 9, 2013 at 17:11
1

You have two problems:

First, don't use quotes in --filter (especially with --protect-args that says use this to avoid having to escape spaces, though my testing here shows that double quotes are taken literally without it too). The filter should be - Application Data/. As you have written it with the quotes it's looking for a file named " in the folder named "Application Data

The other problem is that you want each filter to be a separate argument to rsync, so use @ instead of * in the command:

rsync --dry-run -vv --stats \
      --recursive \
      "$PROTECTARGS"  \
      "${FILTER[@]}"  \
      '/hddOld/Documents and Settings' \
      '/hddNew/'

Tested with:

GNU bash, version 4.1.5(1)-release (x86_64-pc-linux-gnu)
rsync version 3.0.7 protocol version 30

I created an array with arr=(a b "c d"), then used touch "${arr[@]}" to create files

a
b
c d

in dir1.

rsync -r --filter='- c d' dir1/ dir2

copies a and b to dir2.

rsync -r --filter='- "c d"' dir1/ dir3

copies all the files since rsync is looking for a file named "c d" with quotes in the filename. Also tested using mkdir and - c d/ and - "c d/" to make sure directory filters did not work differently, and "c d/" did not exclude the directory while c d/ did exclude it.

2
  • i had initially used @. The --filter='- "Application Data/" works for me when run directly from bash, but not from a script. What rsync / OS are you using?
    – rjt
    May 9, 2013 at 17:07
  • @rjt updated with my test methodology.
    – DerfK
    May 9, 2013 at 22:54

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .