Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have trouble getting all files from an FTP Server. Usually I get all files using the command line/terminal with syntax the following syntax: get a.txt (very, very time wasting with login authentication first)

So, my expectation is I wanna get all file from FTP server, with a script(batch), So I'm not wasting my time using the terminal/command line. I just want to run the script. Of course, login authentication needs to be included in the script. How would I do this?

share|improve this question

3 Answers 3

Save yourself a whole lot of trouble trying to script this with a FTP client. Download ncftp, and use that.

Basically:

ncftpget -R ftp://user:password@host/path/to your files

-R means it does a recursive transfer

share|improve this answer

For this kind of stuff I am using wget:

wget -m ftp://user:password@ftp.example.com

Or lftp's 'mirror` command:

lftp -c mirror ftp://user:password@ftp.example.com/
share|improve this answer

SCP

First, I would recommend that you use SCP over FTP. FTP is insecure and transfers credentials and files in the clear. SCP is, in essence, sc running over ssh and is rather secure, especially if you use keypairs for authentication.

SFTP

Another option is to use SFTP. In spite of the name, it is not FTP over an SSL or SSH connection. It is an entirely new protocol. However, as with the above recommendation, it is not an exact answer to your question, but rather a bit of soapboxing about the insecurity of FTP.

FTP

Choose your ftp client and then research how it handles scripting. You haven't mentioned what OS you are using or which client on your OS you are using. You'll need to provide that information before we can be of much more use to you.

What OS and FTP client are you using?

EDIT: I made the connection that "batch file" equates to a Windows environment. In that case, try this:

ftp.exe -s:filename

The -s feeds the ftp command a file that has FTP commands within it. Within the file that ftp.exe reads, create commands that log in and the retrieve files.

open ftp.server.name
user
[username]
[password]
get *.txt
quit

For more information about FTP and batch files, check out this Microsoft KB article.

If you're using linux, you have a bevy of ftp clients available within your shell of choice. When making the connection to your FTP server, you will have the option of providing the username and password with various switches that the client itself designates. The standard syntax for username and password in an FTP URL is ftp://user:password@host.tld/

From there a new line in your script using the get command with wild cards will download all of the files that you want. For example get *.txt However, be advised that the exact syntax for geting multing files and mirroring directories will depend on your FTP client. My preference is ncftp which has different mirroring options than other FTP clients.

tar

As a bonus, I would recommend first tarring the files that you want to cut down on the size of your transfer as well as preserve UNIX permissions. FTP strips original permissions from files which can be rather annoying when restoring a web site or application.

However, this can't normally be done with an FTP client unless the FTP server itself supports extended commands. For example, the FTP server ncftpd offers some extended commands that other FTP servers do not. So now I've come around full circle to suggesting that you set up an SSH tunnel so that you can tar the directory first. With an SSH daemon set up, you might as well then SCP into the server and retrieve the TAR file rather than using FTP.

If you must keep FTP in the equation, you could create a cron job on the server that tars up the files you want, drops it in your FTP directory and then create an FTP script that connects to the server and retrieves the tar file.

Fin

Automation is your friend. You're on the right track, just start thinking more about security and optimization.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.