Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Is it possible to wget an entire index/parent directory with wget? Because I am trying to transfer images to a VPS of mine from my old shared host.

When I say an index/parent directory, I meant like this:

enter image description here

So basically I want wget to go through each file and download it until all have been downloaded.

Cheers.

share|improve this question
    
just playing devils advocate here...you don't have ftp access to your old shared host? –  iainlbc May 1 '11 at 22:09
    
I agree with @iain, if its your vps can't you just tar/zip or transfer using ftp/sftp/scp? –  coredump May 1 '11 at 22:14
    
I do have ftp access, but I don't know how to do FTP from vps to old host. –  MacMac May 1 '11 at 22:14
    
You can use your desktop as the middleman- Download everything from old server to your desktop, FTP to new server, upload. –  iainlbc May 1 '11 at 22:20
    
Not with my 100kb/s speeds, no. That would take me forever lol. –  MacMac May 1 '11 at 22:21

4 Answers 4

up vote 2 down vote accepted

you did mention there is ftp access, wget actually supports ftp out of the box:

wget -r -l0 --ftp-user=user --ftp-password=pass ftp://yourdomain.com/dir/*
share|improve this answer
    
this is useful, thanks! –  iainlbc May 2 '11 at 16:07

the -r (recursive) wget switch should work, -l X specifies how deep to traverse (5 levels max):

wget -r -l 1 http://domain.com/directory/

source: http://www.gnu.org/software/wget/manual/wget.html#Recursive-Download

share|improve this answer
    
Damn! I knew there was an easier way. Mine's more creative. :) Nicely done! –  jdcarg May 1 '11 at 22:32

Assuing you have no shell or FTP access nor any control panel, and that this is a legitimate request, you might want to have a look at spidering software - I've found pavuk useful for this in the past.

share|improve this answer

Not sure if wget supports this or not, but you could use a bash script to accomplish this fairly easily.

Step 1. Copy and paste all of the text from the "index of /i" into a file called "files" or something else that makes sense on the new host.

Step 2. On your new machine use the following command to do a search & replace to create the wget commands for each file you need to download.

cat files | sed "s/^/wget http:\/\/yoursite.com\/path\//"

This reads the file named "files" and then pipes it to sed which inserts "wget http://yoursite.com/path/file1.png"

For example I ran it on a file on my local computer. The "files" file contains just the .png files like the following:

blah.png
blah1.png
blah2.png

and the output looks like the following after running the command posted above.

wget http://yoursite.com/path/blah.png
wget http://yoursite.com/path/blah1.png
wget http://yoursite.com/path/blah2.png

At that point, you just copy and paste the all the wgets that are generated in terminal or from command line it it will download each file.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.