Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Greetings, I am using ncftpput to transfer a lot of files from on server to the other

ncftpput -f server.txt -vRdb /public_html /var/www/site.com

What happens is, the connection gets cut after about 100 transfers. There are thousands of files that have to be transferred. Problem is after the connection is cut and I reenter the command, it starts from the beginning. Replacing all existing files, making the previous transfer redundant. Is there a way to skip existing files? It's not in the man page so I'm assuming no. So does anybody suggest an alternative command line solution?

share|improve this question
    
Are you stuck with FTP? Seems like rsync would be the best tool to use here. –  innaM Aug 10 '09 at 9:48
    
sorry forgot to mention that the second server, the receiving end is just a shared host, I only have FTP access –  Moak Aug 10 '09 at 9:51
add comment

5 Answers 5

up vote 3 down vote accepted

lftp is a great tool for this, it's scriptable using the mirror command:

 lftp has builtin mirror which can download or update a whole
 directory  tree.  There  is  also reverse mirror (mirror -R)
 which uploads or updates a directory tree on server.  Mirror
 can also synchronize directories between two remote servers,
 using FXP if available.

$ lftp -f

script_file:

open ftp://<user>:<pass>@ftp.blah.org/~
set net:timeout 10
set net:max-retries 2
set net:reconnect-interval-base 5
set net:reconnect-interval-multiplier 1
mirror dir1 dir1
exit
share|improve this answer
    
trying now, still need to reconnect every now and again. Any idea how to automate this? –  Moak Aug 10 '09 at 10:25
    
I have updated my answer –  rkthkr Aug 10 '09 at 10:49
add comment

You could try to work with ncftpput's -DD option. This will delete the local file after it was successfully transfered. If you don't want to delete files in your /public_html directory, you could copy them to a temporary directory first.

share|improve this answer
add comment

Have you tried this option?

-A Append to remote files, instead of overwriting them.

(from ncftp site)

share|improve this answer
add comment

Try wput

share|improve this answer
add comment

Is there any reason why you can't use rsync rather than ftp?

IMO rsync is the right tool for this job. It will only transfer new and changed files, which means that if the transfer dies and has to be restarted, it will pick up from where it left off rather than start from the beginning again.

share|improve this answer
    
Yes, I would have liked to use RSYNC, however I only had ftp access to the second server. And I think RSYNC doesn't use ftp... –  Moak Aug 11 '09 at 11:29
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.