Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have to transfer a directory with lots of small files (250k, avg size 5mb) from one server to another. Both run on Debian Linux. Is there a protocol that is recommendable regarding the transfer of large number of files?

share|improve this question
add comment

3 Answers

up vote 7 down vote accepted

You can tar the files and move the tar. Or pipe tar with SSH: see http://www.cyberciti.biz/faq/howto-use-tar-command-through-network-over-ssh-session/

Tar-ing the files deletes the overhead for each single file as the tarball is actually a single file.

If you can't untar/unzip on the other side (maybe because it's a shared hosting FTP) there's not much you can do.

share|improve this answer
add comment

rsync would certainly fit the bill here as well. Something like

server1$ rsync -av /your/directory server2:/destination/

It's an extremely flexible and powerful tool with many useful options. Run it with the -n flag first to see what it will do when trying to work out the correct syntax.

share|improve this answer
add comment

You do have an option if it's shared hosting and they offer sftp (which any good ones does) and use sshfs to mount the remote sever to your local system. You can then move the files as usual.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.