6

The goal is to rsync (or using any other tools that give the desired result) only files that are created or modified after a date. It should work with filenames containing spaces (and/or other characters that needs to escaped).

I succeeded using this command on OS X 10.6.5: find . -newermt "2010-11-23 16:52:50" -type f -exec rsync -vuptn '{}' ../rsync_test/ ';' I don't think -newermt is working on ohter OS than OS X though.

The problem with this command is that it runs rsync once per file found and that doesn't seems to be the proper way to do it. I tried to change the last ; with a + so it waits for the entire file list, but it brakes (I did try "+",\+ and other way to escape).

Background: I am trying to backup my files at a remote location. The files being a lot photos in raw and other format (photographs work. about 30GB per day), rsync'ing though the network would take unusable time due to small upload bandwidth (ADSL sucks). So the idea is to have a lot of disk space at a remote location and transporting new/modified files on a small external hard drive once a week. rsync can then easily be used to add the new/modified files at the remote location. For that I need to sync the new/modified files on an external hard drive during the week so I always have 2 copies of every files either at a remote location for files older than a week or on an external hard drive for new/modified files since the last transport.

I was going to make this command run every 10 minutes (cron) with the date and time from the last time it was run.

Can you fix my command? Or do you have an even better solution?

Improve this question
1
  • Recent versions of GNU find support -newermt.
    – Dennis Williamson
    Nov 23, 2010 at 19:30

2 Answers 2

Reset to default
9

I'm a little bit confused about what you're trying to do. Rsync, all by itself, will only transfer new/modified files. So if you transport data on a hard drive once/week, you can run rsync periodically and it will no more bandwidth than (and possibly less than) the solution you're proposing using find.

All this work with find seems like you're just duplicating features that are already there.

If you really want to use find, you don't need xargs or xargs-like features; just do this:

find ... -print | rsync ... --files-from=-

Where ... is "whatever other options you feel are appropriate". This causes rsync to read a list of files from stdin.

Improve this answer
4
  • The idea is to empty the "transport" hard drive every time it's data has been synchronized with the remote storage hard drives. So rsync (as I understand) will copy the files already on the remote hard drive since it only has access to the transport drive.
    – Yanik
    Nov 23, 2010 at 18:43
  • Ah, I see. So the find | rsync solution here will do what you want. You're not getting much benefit from rsync here (since you're always copying to an empty destination rsync doesn't get to do any of its fancy bandwidth-reducing work); you may also want to look into something like the GNU version of tar, which has good support for incremental backups (which is what you're doing). See, for example, gnu.org/software/tar/manual/html_section/Incremental-Dumps.html.
    – larsks
    Nov 23, 2010 at 18:49
  • Using find . -newermt "2010-11-23 16:52:50" -print0 | rsync -vuptn -0 --files-from=- ../rsync_test/ gives me a syntax or usage error. Same as if I use -print
    – Yanik
    Nov 23, 2010 at 18:50
  • 2
    I found out. You still need to specify a source for rsync. The working command is then: find . -newermt "2010-11-23 16:52:50" -print0 | rsync -vuptn -0 --files-from=- . ../rsync_test/
    – Yanik
    Nov 24, 2010 at 9:12
1

+ only works with GNU find, and OSX has BSD find. Use -print0 and xargs -0 to simulate it.

-newermt should work for new enough versions of GNU find. My Fedora 13 machine (GNU find 4.4.2) has it, but not my CentOS 5 machine (GNU find 4.2.27). You should be able to simulate it by using touch to create a file with the appropriate time and then use -newer on it.

Improve this answer
3
  • I already tried with xargs, but can't seem to find out how I should use it. Can you give me an example?
    – Yanik
    Nov 23, 2010 at 18:54
  • + was taken from the man on my machine :) but didn't work.
    – Yanik
    Nov 23, 2010 at 18:55
  • 1
    find ... -print0 | xargs -0 -I{} ... rsync {} <destination>
    – Ignacio Vazquez-Abrams
    Nov 23, 2010 at 18:58

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .