Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I have a 1TB folder with many files on Centos server I need to make sure another Centos server has the same folder and files for backup. right now the backup server has no files. once I copy all the folders I guess I can use "rsync" but for the first copy I think I'll need something else I have 2 NIC on both servers in cross cable which I would like to use for the backup. any good idea? thanks

share|improve this question
    
+1 for all the rsync mentions. This is what this tool has been invented for. –  Axel Knauf Apr 2 '11 at 18:03

6 Answers 6

you can also tar and untar over ssh in a single command, this is good for only one trancefer you can use rsync after you do a full copy

cd /data ; tar cf - . | ssh -C -c blowfish root@server2 "cd /data ; tar xf - "

I am not sure, but it could be faster then rsync when there are a lot of small files

share|improve this answer
    
what about windows??? –  Jeson Park Feb 15 at 18:33

There is nothing wrong with using rsync for that - if it takes that long, just start it on a Friday ;) .

In some cases, if there are a lot of small files and the latency is high, it might be advisable to tar the whole tree, copy the tarball and untar it at the target system. For good measure, you can also calculate checksums of the tarball on both systems.

share|improve this answer

rsync is perfectly fine for initial backup too. If you have two NICs and slow ethernet switches (or those servers are on different networks), use crossover cables and rsync. Additionally, if you have something performance critical going on, using crossover cable on secondary NIC is good idea, as it offloads traffic to separate card.

If you have 1Gbit/s network cards, then it's actually much faster to use rsync than buying external disk.

Already mentioned solution with tar - if there is any glitch, it'll fail, and you have to start over from the beginning.

I really, really recommend using rsync,

rsync -avz --progress /srv/source -e ssh username@destination.server:/srv/destination

Even if you have 100Mbit/s network, using USB disks is bad solution - it's something like 30MB/s for both writing and reading - 15MB/s in average, plus you have to manually move it and run same commands again. 100Mbit/s is about 10MB/s (some network overhead included).

share|improve this answer

Do rsync from different sessions simultaneously:

rsync -avz --progress --stats /home/[a-l]* root@remotebkpserver:/backup/home

rsync -avz --progress --stats /home/[m-z]* root@remotebkpserver:/backup/home

rsync -avz --progress --stats /var root@remotebkpserver:/backup
share|improve this answer

1000M ethernet nics doesn't require cross cable for peer-to-peer connection. you can use standard cat5 cable to connect between those servers. then you can use 1000M instead of 100M on the crossed cable.

share|improve this answer

Buy a 1Tb disk. Plug it into server 1, preferably using SATA or eSATA, although USB is fine too. Copy all the files to it. Plug it into server2. Your files have now arrived. Much faster than rsync.

Here's a picture I took today while doing this very thing: 1Tb drive in colo box

share|improve this answer
    
I think this will take longer. You have to copy the data to the transfer drive on server1 and then back onto the normal drive on server2. With gigabit ethernet, you can expect data rates that are much higher than USB and likely more than 50% of (e)SATA. Also, you have to watch this and go to the server to switch drives after a few hours. –  Sven Apr 2 '11 at 11:56
    
@Sven, you don't have to copy at the end, just leave the new drive in server2. –  Paul Tomblin Apr 2 '11 at 12:04
    
Funny I got all these downvotes on the very day I just brought 2 1TB drives out to install on my 1U server on a colocation facility because it was the fastest way to get that volume of data pushed between two servers. –  Paul Tomblin Apr 2 '11 at 21:18
    
well, if you can plug crossover cable between two of your servers (and 1Gbit/s NICs), no, it's not. But if your servers do have slow connection and those are far away, then it might be. –  Olli Apr 3 '11 at 6:02

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.