I have to transfer a directory with lots of small files (250k, avg size 5mb) from one server to another. Both run on Debian Linux. Is there a protocol that is recommendable regarding the transfer of large number of files?
You can tar the files and move the tar. Or pipe tar with SSH: see http://www.cyberciti.biz/faq/howto-use-tar-command-through-network-over-ssh-session/
Tar-ing the files deletes the overhead for each single file as the tarball is actually a single file.
If you can't untar/unzip on the other side (maybe because it's a shared hosting FTP) there's not much you can do.
rsync would certainly fit the bill here as well. Something like
server1$ rsync -av /your/directory server2:/destination/
It's an extremely flexible and powerful tool with many useful options. Run it with the
-n flag first to see what it will do when trying to work out the correct syntax.
You do have an option if it's shared hosting and they offer sftp (which any good ones does) and use sshfs to mount the remote sever to your local system. You can then move the files as usual.