Sign up ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I want to copy all of the files and folders from one host to another. The files on the old host sit at /var/www/html and I only have FTP access to that server, and I can't TAR all the files. Regular connection to the old host through FTP brings me to the /home/admin folder.

I tried running the following command form my new server:

wget -r

But all I get is a made up index.html file.

What the right syntax for using wget recursively over FTP?

share|improve this question
Note that, by default, wget -r has a maximum recursion depth of 5; if you have an deeper subdirectories, they will be ignored unless you change this (see -l in the man page). – mikewaters May 23 '11 at 19:21

13 Answers 13

Try -m for --mirror

wget -m
share|improve this answer
For security reason, it would be good to use --ask-password to avoid password being kept in history. – checksum Oct 4 '14 at 2:49
This is so cool. Worked like a charm.. – Sambhav Sharma Oct 7 at 21:11

You have it right, you just need a trailing * on the end:

wget -r ftp://username:password@*
share|improve this answer
This would only go one directory deep.Better to use the -m flag – Svenn Dhert Aug 13 at 12:18

besides wget, you may also use lftp in script mode. the following command will mirror the content of a given remote FTP directory into the given local directory, and can be put into the cron job:

lftp -c 'open <hostname>; user <username> <password>; mirror -e <remote-src-path> <local-dest-path>; quit'

it automatically handler recursion into directories and allows specifying the remote source starting directory from to download data from.

share|improve this answer

You can use 'curlftpfs - mount a ftp host as a local directory' and, once mounted, you can use normal file tools like 'cp -r'.

share|improve this answer
curlftpfs has major performance problems right now unless you downgrade libcurl3-gnutls – Hengjie Feb 6 at 5:40

Should work try:

wget -r
share|improve this answer

Use -m rather than -r, because of the recursion depth.

share|improve this answer


wget -m

and it will mirror all the files and folders.

share|improve this answer
As I said, logging in through FTP takes me to the /home/admin folder, and the files I need are in /var/www/html So when I run the command you suggested, I only get the contents of /home/admin I tried running it with, but then it tries to CWD /home/admin/var/www/html. How do I make it go to this folder from the root? – user9406 Jun 13 '09 at 17:23
I have no experience with this particular problem, but you could try making a symlink to /var/www/html under your home. Then you could use an address like – prestomation Jun 13 '09 at 17:29
I tried creating a symlink, but it resulted with wget creating a similar symlink on my local. – user9406 Jun 13 '09 at 17:32

That's the right syntax. Not sure why you aren't getting the expected results.

As ever there is more than one way to do it. Try ncftp, in particular ncftpget

share|improve this answer
I can't install more software on my new server. Shouldn't I be telling wget to download all files from /var/www/html? I tried wget -r but I got a directory not found error. – user9406 Jun 13 '09 at 9:37

I can understand if you're trying to dump this into cron or something, but why not simply ftp into the server with your normal client and mget *? This might be a quicker path to success.

share|improve this answer
I tried mget * but it didn't work with sub-folders, saying the local sub-folder doesn't exist. Is there a way to make him create the local folders automatically? – user9406 Jun 13 '09 at 17:16
Depending on your client, the -r switch usually does the trick. IE: mget -r * – dr.pooter Jun 15 '09 at 6:17
mmm. AFAIK standard ftp client in linux is not designed to retrieve directories recursively. I mean - there is no -r option. other clients like ncftp or lftp support recursive retrieval but they usually not available by default. – Stann Feb 13 '11 at 5:42

As I said, logging in through FTP takes me to the /home/admin folder, and the files I need are in /var/www/html

I think this will work in your case:

wget -r
share|improve this answer

Check below wget command to download data from FTP recursively.

wget --user="" --password="" -r -np -nH --cut-dirs=1 --reject "index.html*" ""

-r : is for recursively download.

-np : is for no parent ascending.

-nH : is for disabling creation of directory having name same as URL i.e.

--cut-dirs : is for ignoring no. of parent directories. Value of this option will differ for your command.

You can check by executing above command.

share|improve this answer

I came across a windows hosting with the username as brinkster/username . so wget will throw an error if you use the syntax :

wget -m ftp://brinkster/ 

to get past this ,

wget -m 

I had to face this with a windows hosting @ brinkster .

share|improve this answer

wget --user username --password yourpassword will do the job or you can add * instead of filename entring : makes wget think of port which is invalid in your case.

share|improve this answer
This answer to a four year old question with multiple upvoted answers doesn't offer anything new and even omits the requirement for a recursive copy. – Sven Jun 20 '13 at 0:10

protected by Sven Jul 24 at 7:05

Thank you for your interest in this question. Because it has attracted low-quality answers, posting an answer now requires 10 reputation on this site.

Would you like to answer one of these unanswered questions instead?