Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I need to copy a 700G ISCSI mounted folder to a folder on the local hard drive, while maintaining the existing permissions and ownership. The data in the folder consists of about a million mostly small files, and the new directory needs to exactly match the permissions and ownership of the original.

Using rsync -az /original_folder/ /new_folder gets all the permissions correct, but it has been over an hour "scanning for files", and has not even started coping over yet. The data is openvz disk shares, and there have been permission problems in the past when they were copied over using "cp".

Is there a faster way to copy a huge number of small files while exactly preserving the permissions and ownership? Maybe some rsync flags that can avoid the initial scan? Ideally a tool that can pick up where you left off if the process is interrupted.

share|improve this question
    
not that you can change it at this moment, but i remember that moving from ext3 to xfs reduced the 'scanning for files' time on a 6TB volume from 23 minutes to less than 15 secs. (now it's at 17TB, and still less than 30secs for scanning) –  Javier Jul 15 '11 at 11:29
add comment

4 Answers 4

Have you considered using tar? Something along the lines of tar -cpOC <sourcedir> . | tar -xpC <targetdir> -f - might do you.

share|improve this answer
add comment

What version are rsync are you using? Version 3 handles small files quite a bit better. You can also use -P to get a sense for how far along you are.

As for recovering from partials.. hrm. You could go into your root dir and run something like..

for i in * ; do rsync -azP $i /newfolder ; done

then skip any subdirs that you've already copied

share|improve this answer
add comment

In this situation you should probably try to duplicate the volume rather than trying to duplicate the individual files.

Does your iSCSI target have a way to clone the volume?

share|improve this answer
add comment

cp -aux (as root) should do the trick.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.