Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm building a service where I'll need to store a ton of files on a filesystem. I've heard in the past that it's better to separate up files by folder when dealing with large amounts of files. Is this really accurate for modern Linux filesystems such as ext3, ext4, and btrfs? If so, how much should I divide up files? Should I keep 1000 files to a folder?

The current setup I have divides them up into hundreds, thousands, tens of thousands, and hundreds of thousands like so:

stream id: 123456

stream id: 654321

stream id: 5

Does this make sense for my application?

share|improve this question
Just because they're all "modern", doesn't mean they all perform the same in all circumstances. Please be more specific in future. – womble Aug 12 '11 at 1:15
up vote 5 down vote accepted

You should also consider the amount folders you are creating.

In the related question the general rule seems to be up to 20k files per directory for performance reasons:
Maximum number of files in one ext3 directory while still getting acceptable performance?

share|improve this answer
I suppose I'll filter down to maybe a thousand or ten thousand per folder, as that seems to be a good median. – Naftuli Tzvi Kay Aug 12 '11 at 1:43

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.