Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

What is the best way to automate the ftp process. I mean can a script be written which when executed would ftp to a remote system and fetch files with out human intervention.

Please suggest.

share|improve this question

11 Answers 11

I personally use lftp, which is highly scriptable (see this tutorial).

Of course, you can also write an adapted FTP client using perl/ruby/python/your-prefered-scripting-language.

share|improve this answer
    
I agree. lftp is the best tool to use. Even supports parallel ftp processes and automation of sftp. –  fpmurphy1 Aug 18 '09 at 14:05

I think you can use wget to do such a thing:

WGET is free so you can grab that and check the help file

Something like:

wget -user=username -pass=password ftp://ftp.moose.com/download.zip

Obviously usable with variables in a script etc.

Just noticed this is for solaris, so not sure if this would work.

share|improve this answer
    
It might be installed in /usr/sfw/bin, which is probably not in your search path by default. If you can't find it there, look on sunfreeware.com. –  Barry Brown Aug 18 '09 at 10:33

On Solaris you can also use expect automation tool. Of course you should install it first. For FTP you can find an example in Wikipedia article on Expect.

share|improve this answer
    
expect works on several platforms –  warren Aug 19 '09 at 22:53

Hi I've written one once for our needs,

#!/bin/sh

hostname="IP-ADRESS"

port="21"

username="serverfault"

password="serverfalut"

storage_path="DESTINATION"

current_date="$(date +%Y%m%d)"

$remotefile="REMOTE-FILE"

ftp -in $hostname $port << EOF

quote user $username

quote pass $password

binary

get $remotefile $storage_path/

delete $output_dir/output$current_date.tar.gz

quit

close

EOF

exit 0

share|improve this answer

There are Perl packages for this purpose. For instance CPAN modules Net::Lite::FTP and Net::FTP.

It will allow your script to make decisions based on information from the server. E.g. if the file name is not constant (may contain a date or a version number) then the script can process a directory listing before deciding which file to download.

I have succesfully used it for automating downloading/updating of large molecular sequence databases.

share|improve this answer

ncftp usually includes ncftpget and ncftpput, which would be suitable for inclusion in scripts. They can usually be built from source, or check csw or sunfreeware.

share|improve this answer

The Solaris ftp command can be automated with a netrc(4) file. The macdef init is the key. sample:

cat ~/.netrc
machine remote-host1.domain.com login my-ftp-account
password password4host1
macdef init
cd
/pub/DOWNLOAD get readme

default login anonymous password my-email@mail.domain.com

Notes:

  1. The two blank lines after the init macro definition are important,
  2. This also automates an interactive ftp session to remote-host1.

Your script can create the netrc file before it calls ftp.

share|improve this answer

I like curl the best:

curl is a command line tool for transferring files with URL syntax, supporting FTP, FTPS, HTTP, HTTPS, SCP, SFTP, TFTP, TELNET, DICT, LDAP, LDAPS and FILE. curl supports SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload, proxies, cookies, user+password authentication (Basic, Digest, NTLM, Negotiate, kerberos...), file transfer resume, proxy tunneling and a busload of other useful tricks.

Very handy for all your URL-fetching needs.

share|improve this answer

I typically just use ftp with an authentication file .netrc and then cat everything into it. For example

cat >> EOF | ftp
open ftpserver
mget *
bye
EOF
share|improve this answer

You've gotten a lot of recommendations already to use something different than FTP. That's fair, because scripting FTP is a little annoying, but it's not necessarily portable, nor is it teaching you anything new.

It's actually quite simple to script by directing commands to ftp from STDIN. This sort of thing works for many kinds of tools.

cd /target
ftp -n host <<EOF
USER falken
PASS joshua
cd place
binary
get file
bye
EOF
echo 'hi! um... what'\''s this all about again? I am done.'

The login part could be automated by adding information to the script owner's ~/.netrc file. In fact, if you man netrc you'll learn a lot about automating ftp.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.