Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

Possible Duplicate:
Quickest way to transfer 55GB of images to new server

I am migrating from one server to another. I want to transfer the thousands of file from one server to the other.

What is the best approach here?

Both system is running CentOs. The one I am migrating from uses DirectAdmin, the other cPanel. I have SSH-access on both.

I am currently taring the different folders, but when the files are large and amount of file are big it takes too long)

share|improve this question

marked as duplicate by Iain May 28 '12 at 18:32

This question has been asked before and already has an answer. If those answers do not fully address your question, please ask a new question.

    
serverfault.com/questions/208300/… –  gparent May 28 '12 at 17:35

5 Answers 5

up vote 6 down vote accepted

From the source host:

$ rsync -avz --progress /path/to/files/ user@host2.example.com:/path/to/files/
share|improve this answer
1  
I would try without compression (-z) first. I have noticed that it actually slows down the speed of a transfer in high bandwidth environments such as a LAN. –  Soviero May 28 '12 at 16:40

I can tell from my experience that tar over ssh is faster than rsync when dealing with a large number of small files.

I suggest to give it a try.

The command was something like this (it preserves the permissions):

tar cXpf - /data | ssh user@new-server "tar xpf - -C /new-location/"

The reason could be the fact that tar creates a contiguous stream and ssh is compressing it.

Try and tell us the results.

share|improve this answer

I would look at using rsync with compression between the two servers, using SSH as a "tunnel" to make the connection over the network.

share|improve this answer

RSync over SSH would probably be your best bet. Have a look here on how it can be setup.

share|improve this answer

The best way is to use rsync, but why?

The reason is, while you're transferring the large chunk of files and the connection drops for some reason, you don't need to worry about what files were transferred and what were not.

This will be taken care by rsync as it'll quickly compare the files present in the destination with the source and transfer over only the relevant ones. This approach saves both bandwidth and time.

You can use the following syntax for rsync

rsync -avz /path/to/source user@host.com:/path/to/destination

Apparently, if you've a SSH key and password authentication is disbaled on the destination server, use this

    rsync -avz /path/to/source -e "ssh -i /path/to/ssh-key" user@host.com:/path/to/destination
share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.