Sign up ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I want to copy all of the files and folders from one host to another. The files on the old host sit at /var/www/html and I only have FTP access to that server, and I can't TAR all the files. Regular connection to the old host through FTP brings me to the /home/admin folder.

I tried running the following command form my new server:

wget -r

But all I get is a made up index.html file.

What the right syntax for using wget recursively over FTP?

share|improve this question
Note that, by default, wget -r has a maximum recursion depth of 5; if you have an deeper subdirectories, they will be ignored unless you change this (see -l in the man page). – mikewaters May 23 '11 at 19:21

13 Answers 13

Check below wget command to download data from FTP recursively.

wget --user="" --password="" -r -np -nH --cut-dirs=1 --reject "index.html*" ""

-r : is for recursively download.

-np : is for no parent ascending.

-nH : is for disabling creation of directory having name same as URL i.e.

--cut-dirs : is for ignoring no. of parent directories. Value of this option will differ for your command.

You can check by executing above command.

share|improve this answer


wget -m

and it will mirror all the files and folders.

share|improve this answer
As I said, logging in through FTP takes me to the /home/admin folder, and the files I need are in /var/www/html So when I run the command you suggested, I only get the contents of /home/admin I tried running it with, but then it tries to CWD /home/admin/var/www/html. How do I make it go to this folder from the root? – user9406 Jun 13 '09 at 17:23
I have no experience with this particular problem, but you could try making a symlink to /var/www/html under your home. Then you could use an address like – prestomation Jun 13 '09 at 17:29
I tried creating a symlink, but it resulted with wget creating a similar symlink on my local. – user9406 Jun 13 '09 at 17:32

wget --user username --password yourpassword will do the job or you can add * instead of filename entring : makes wget think of port which is invalid in your case.

share|improve this answer
This answer to a four year old question with multiple upvoted answers doesn't offer anything new and even omits the requirement for a recursive copy. – Sven Jun 20 '13 at 0:10

Use -m rather than -r, because of the recursion depth.

share|improve this answer

You have it right, you just need a trailing * on the end:

wget -r ftp://username:password@*
share|improve this answer
This would only go one directory deep.Better to use the -m flag – Svenn Dhert Aug 13 at 12:18

I came across a windows hosting with the username as brinkster/username . so wget will throw an error if you use the syntax :

wget -m ftp://brinkster/ 

to get past this ,

wget -m 

I had to face this with a windows hosting @ brinkster .

share|improve this answer

As I said, logging in through FTP takes me to the /home/admin folder, and the files I need are in /var/www/html

I think this will work in your case:

wget -r
share|improve this answer

You can use 'curlftpfs - mount a ftp host as a local directory' and, once mounted, you can use normal file tools like 'cp -r'.

share|improve this answer
curlftpfs has major performance problems right now unless you downgrade libcurl3-gnutls – Hengjie Feb 6 at 5:40

besides wget, you may also use lftp in script mode. the following command will mirror the content of a given remote FTP directory into the given local directory, and can be put into the cron job:

lftp -c 'open <hostname>; user <username> <password>; mirror -e <remote-src-path> <local-dest-path>; quit'

it automatically handler recursion into directories and allows specifying the remote source starting directory from to download data from.

share|improve this answer

I can understand if you're trying to dump this into cron or something, but why not simply ftp into the server with your normal client and mget *? This might be a quicker path to success.

share|improve this answer
I tried mget * but it didn't work with sub-folders, saying the local sub-folder doesn't exist. Is there a way to make him create the local folders automatically? – user9406 Jun 13 '09 at 17:16
Depending on your client, the -r switch usually does the trick. IE: mget -r * – dr.pooter Jun 15 '09 at 6:17
mmm. AFAIK standard ftp client in linux is not designed to retrieve directories recursively. I mean - there is no -r option. other clients like ncftp or lftp support recursive retrieval but they usually not available by default. – Stann Feb 13 '11 at 5:42

Try -m for --mirror

wget -m
share|improve this answer
For security reason, it would be good to use --ask-password to avoid password being kept in history. – checksum Oct 4 '14 at 2:49
This is so cool. Worked like a charm.. – Sambhav Sharma Oct 7 at 21:11

Should work try:

wget -r
share|improve this answer

That's the right syntax. Not sure why you aren't getting the expected results.

As ever there is more than one way to do it. Try ncftp, in particular ncftpget

share|improve this answer
I can't install more software on my new server. Shouldn't I be telling wget to download all files from /var/www/html? I tried wget -r but I got a directory not found error. – user9406 Jun 13 '09 at 9:37

protected by Sven Jul 24 at 7:05

Thank you for your interest in this question. Because it has attracted low-quality answers, posting an answer now requires 10 reputation on this site.

Would you like to answer one of these unanswered questions instead?