Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I need to chown 1.5 million files on a drive. I'm currently doing:

sudo chown -R www-data:www-data /root-of-device

but it takes an awfully long time to run. I was wondering if there was some sort of superfast low-level way to chown every file on the drive.

share|improve this question

3 Answers 3

up vote 7 down vote accepted

Use xargs -p or GNU parallel to speed things up considerably.

share|improve this answer
1  
On a single filesystem, the speedup will be less than you think; potentially it could be slower because of contention between the multiple threads of execution accessing the same filesystem. –  geekosaur Apr 10 '12 at 21:24
1  
That should not be the case in XFS, as it supports parallel meta data operation according to the number of allocation groups created at formatting time. A parallel chmod will be faster than the sequential version. –  pfo Apr 10 '12 at 21:59
    
I tried xargs -P4 (number of allocation groups on my xfs) and it ran slightly faster than the regular chown. I ran it a few times though and I think the difference in timings was just AWS EBS i/o fluctuations. –  Aidan Kane Apr 11 '12 at 13:36

Unfortunately I do not think there is such a thing, but I would be pleasantly surprised if there was. You could write your own implementation in C and optimise it heavily. However the success of that depends on how well optimized chown is to begin with. And considering it's one of the core utilities I would say it's rather optimized. In addition you are bound most likely by i/o speed.

I have had some success avoiding limitations of ls and rm by piping the results of find to xargs, in the case a directory has a lot of files, i.e.:

find /path/* | xargs rm

So, a wild guess, maybe this can speed up chown, in case it is slower at recursively scanning a filesystem than find:

sudo find /path/* | xargs chown www-data:www-data
share|improve this answer
    
Ha! Don't fancy my chances of coming up with a better implementation to be honest. It's an Amazon EBS device which means that i/o performance is a bit unstable. –  Aidan Kane Apr 10 '12 at 21:09
    
I updated my answer with a few suggestions that may help. –  aseq Apr 10 '12 at 21:13
    
In theory you could abuse xfs_metadump and xfs_db to get something slightly faster; in practice it's not worth the effort or the potential failure modes. –  geekosaur Apr 10 '12 at 21:21
    
Yeah I think whichever way you look at it it will take a long time... –  aseq Apr 10 '12 at 23:32
    
Thanks for the hints. Playing around with various things it really looks like I'm not going to do much better than regular chown. –  Aidan Kane Apr 11 '12 at 13:38

I'm using Amazon EC2 as well and had this issue. 2 things:

Fixing Current situation: You will have to deal with the slowness. Perhaps you can use "screen" or something similar so that the process can continue in the background.

Fixing Future situation: You could look upstream and see how the files are being generated in the first place. Since you mentioned www-data, I'm assuming that the consumer of the files is Apache. If files are being dropped from another program (NFS, Samba, SSH etc) make sure that those programs are setting the user:group as www-data:www-data.

share|improve this answer
    
screen(1) is a solution, but perhaps better is to use nohup(1). –  vonbrand Feb 19 '13 at 23:13

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.