Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I run Ubuntu Linux and use backup script that copies things over using rsync:

rsync -rc /home/user/source /media/nas/destination

or

rsync -r /home/user/source /media/nas/destination

However - it takes approximately 2.5 hours every day (i run this every day), to get it done. There is approximately 76 Gb of data over 1Gbit lan. So - basically - rsync just doesn't skip existing files in destination - it just copies everything over and over again.

I assume that the problem lies in NAS (d-link 321) and that it's a samba share that doesn't support unix persmission system - right? So this way rsync can't distinguish old files from new files - right? I've tried rsync with -c parametr (skip based on checksum, not mod-time & size) but still no dice, still takes 2.5 hours.

Can anybody suggest the way of syncing appx 70-80 Gb of data that doesn't ivolve "dumb" copying of everything every time?

Thanks:)

UPDATE

well. no luck. I've erased destination directory and and ran this twice:

rsync -r --times /home/user/source /media/nas/destination

It still took 2.5 hours both times. so it copies everything twice.

mmm. I went deeper to investigate this problem. And it looks like rsync doesn't preserve times. I've ran stat command on some files from source and destination - and Access, Modify, Change times aren't preserved on destination. I have no clue as to why does this happen... Any other suggestions? P.s. I relatively new to Linux - sio it can be something really dumb and silly:)

UPDATE 2 Resolved: problem as I realized stemmed from me mounting shares incorrectly. I've been mounting them like this (in /etc/fstab):

//192.168.10.199/Volume_1 /media/Volume_1 cifs   guest,rw,iocharset=utf8,file_mode=0777,dir_mode=0777 0 0

And this is what worked:

//192.168.10.199/Volume_2 /media/Volume_2 cifs    
username=sam,password=some_password,uid=developer,gid=developer,_netdev 0 0

Rsync preserves times now and only takes 3 minutes!

share|improve this question
add comment

4 Answers 4

up vote 5 down vote accepted

Why don't you try:

rsync -a /home/user/source /media/nas/destination

-a is convenience option specifically for archiving. It works for me all the time.

Here is what man page says:

-a, --archive archive mode; same as -rlptgoD (no -H)

share|improve this answer
    
+1 for -a which actually includes the -t –  egorgry Apr 22 '11 at 1:37
3  
-a may cause problems if the NAS doesn't permit assigning permissions, owners, or groups appropriately. –  justarobert Apr 22 '11 at 3:32
add comment

You need to add -t (or --times) to your rsync command-line, so that rsync will preserve file modification times at the destination. That will allow subsequent rsync runs to skip all files with identical sizes and modification times. Also, don't bother with the -c option, unless you know for certain that your destination does not (or cannot) maintain accurate file modification times, for some reason.

share|improve this answer
add comment

You very likely want to be using the -u flag, which tells rsync not to send files that exist in the destination and that either have newer modified times in the destination or have equal modified times and sizes in both places.

share|improve this answer
add comment

In addition to using the -a flag, you should also check the man page for other options. The problem with using a simple mirror backup strategy, is that any corrupted file will be copied over and corrupt your backup as well. One of the rsync options is to backup a copy of the changed files on the destination side, which is much safer.

I highly recommend to take a look at BackupPC as well. It uses rsync and has Debian/Ubuntu packages. It will handle incrementals properly, collating identical files, and compressing everything to save space.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.