up vote 0 down vote favorite
4
share [g+] share [fb]

I am dealing with hundred million files in a filesystem (distributed among a lot of subdirectories), and I need to be able to list them very quickly, particularly in order to rsync them efficiently.

On a other hand, I don't really need to have the actual content of the file kept in cache.

I am constantly adding and removing files, but not that frequently (something like ten times per second).

Is there a way I can tell the OS (2.6.18-194.el5) to use the 24GB available RAM more on inode caching than on file caching? I already looked at /proc/etc/vm/vfs_cache_pressure but it doesn't seem to be exactly what I am looking for...

link|improve this question
feedback

1 Answer

up vote 2 down vote

you can use these 2 commands to do the same job.

Updatedb (to update the list of file and folders location in whole drive)

locate / (to list all files in the whole OS, which is lightening fast as it picks them up from the Database)

link|improve this answer
Yup, I run updatedb from cron to keep the inode cache 'warm.' It works well, take a look at your slabtop to see statistics. Also, running strace -c updated can give you some insight as far as how much gets updated. – Marcin Dec 6 '11 at 13:00
Thank you, but how does that help me improving rsync performance?edit: Ok, just read Marcin's comment, but regarding the warmness of the cache, how is it different from running a simple find /? – john.doe Dec 6 '11 at 13:03
In some implementations, the updatedb shell script actually runs find / to get the initial file list. On Mac OS X it still works this way. – Ladadadada Dec 6 '11 at 14:38
feedback

Your Answer

 
or
required, but never shown
discard

Not the answer you're looking for? Browse other questions tagged or ask your own question.