Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I am trying to mirror most of a ftp sever using wget.

This particular sever keeps old copies of the data files in a folder called 'backup' inside every subfolder. e.g. '/MaffiaOffShore/backup' '/VeryVeryDodgy/backup', which I don't want but my attempts at using --reject 'backup' and using --exclude-directories 'backup' and various other combinations, all have the same result, the whole sever is mirrored, including the backup folder.

Is this a bug or am I missing something?

share|improve this question

3 Answers 3

up vote 0 down vote accepted

The exclude directories option is:

--exclude-directories=List

not --exclude-directory as you have included in your question

from the wget man page

-X list
--exclude-directories=list
   Specify a comma-separated list of directories you wish to exclude
   from download (@pxref{Directory-Based Limits} for more details.)
   Elements of list may contain wildcards.
share|improve this answer
    
typo, but the correct version doesn't work either :-( –  WgetMonkey Jun 22 '09 at 9:12
    
I've just updated my answer '*/backup', which was initially not 100% correct –  kubanczyk Jun 22 '09 at 9:18
    
That works now, thanks. Not the most 'user-friendly' way of writing the software... –  WgetMonkey Jun 22 '09 at 9:32

If you want to exclude /MaffiaOffShore/backup from /MaffiaOffShore/, try

-x/MaffiaOffShore/backup

or,

--exclude-directories=/MaffiaOffShore/backup

explicitly.


So, from the base directory you exclude all un-required directories as,

-x/MaffiaOffShore/backup,/VeryVeryDodgy/backup
share|improve this answer

Yes, your attempts fail also on my wget. However, I've just verified that the following works nicely for all nested subdirectories:

wget -r ftp://user@xxx --exclude-directories '/backup,*/backup,*/*/backup,*/*/*/backup'

The wildcarded '*/backup' will not match /backup but it will match paths like /something/backup. The wildcarded '*/*/backup' will match '/something/else/backup', etc.

share|improve this answer
    
Just tried that, didn't work. One combination I tried gave me a backtrace and crashed! I'm using version 'GNU Wget 1.11.4', what were you using? Are you sure that backup folders existed on the sever you tested this on? –  WgetMonkey Jun 22 '09 at 9:24
    
Mine is wget 1.10.2 so it's older. I've created a test directory tree with backup folders all over it. I guess the problem might be that your tree contains a very large number of directories, so the /*/ expands to a big list in the memory? –  kubanczyk Jun 22 '09 at 9:54

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.