Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

Is it possible to wget an entire index/parent directory with wget? Because I am trying to transfer images to a VPS of mine from my old shared host.

When I say an index/parent directory, I meant like this:

enter image description here

So basically I want wget to go through each file and download it until all have been downloaded.


share|improve this question
just playing devils advocate don't have ftp access to your old shared host? – iainlbc May 1 '11 at 22:09
I agree with @iain, if its your vps can't you just tar/zip or transfer using ftp/sftp/scp? – coredump May 1 '11 at 22:14
I do have ftp access, but I don't know how to do FTP from vps to old host. – MacMac May 1 '11 at 22:14
You can use your desktop as the middleman- Download everything from old server to your desktop, FTP to new server, upload. – iainlbc May 1 '11 at 22:20
Not with my 100kb/s speeds, no. That would take me forever lol. – MacMac May 1 '11 at 22:21
up vote 2 down vote accepted

you did mention there is ftp access, wget actually supports ftp out of the box:

wget -r -l0 --ftp-user=user --ftp-password=pass*
share|improve this answer
this is useful, thanks! – iainlbc May 2 '11 at 16:07

the -r (recursive) wget switch should work, -l X specifies how deep to traverse (5 levels max):

wget -r -l 1


share|improve this answer
Damn! I knew there was an easier way. Mine's more creative. :) Nicely done! – jdcarg May 1 '11 at 22:32

Assuing you have no shell or FTP access nor any control panel, and that this is a legitimate request, you might want to have a look at spidering software - I've found pavuk useful for this in the past.

share|improve this answer

Not sure if wget supports this or not, but you could use a bash script to accomplish this fairly easily.

Step 1. Copy and paste all of the text from the "index of /i" into a file called "files" or something else that makes sense on the new host.

Step 2. On your new machine use the following command to do a search & replace to create the wget commands for each file you need to download.

cat files | sed "s/^/wget http:\/\/\/path\//"

This reads the file named "files" and then pipes it to sed which inserts "wget"

For example I ran it on a file on my local computer. The "files" file contains just the .png files like the following:


and the output looks like the following after running the command posted above.


At that point, you just copy and paste the all the wgets that are generated in terminal or from command line it it will download each file.

share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.