What's the maxium number of files a Unix folder can hold?

I think it will be the same as the number of files.

migrated from stackoverflow.com Aug 27 '09 at 9:30

This question came from our site for professional and enthusiast programmers.

  • 3
    A much better question might be: How many should I use? stackoverflow.com/questions/466521/… – Joachim Sauer Jan 26 '09 at 1:24
  • 1
    I'd love for my site url's to look like site.com/username/ and so on, but thinking that (if im lucky) get more than 2 million users that'd be more than 2 million folders, since I don't want to use a script such as PHP with a modrewrite i was looking at the other possibilitie of folders in a folder – PERR0_HUNTER Feb 5 '09 at 15:41
  • 1
    Do yourself a favor and create subdirectories with a rewriting scheme. – Peter Eisentraut Jan 21 '10 at 12:09

Varies per file system, http://en.wikipedia.org/wiki/Comparison_of_file_systems

  • 9
    I liked that table better when it had a column to tell you if the file system would murder your wife or not. – Paul Tomblin Jan 26 '09 at 1:49
  • lol ! – Zach Jan 26 '09 at 2:39
  • 2
    en.wikipedia.org/w/… – Zach Jan 26 '09 at 2:44

On all current Unix filesystems a directory can hold a practically unlimited number of files. Whereas "unlimited" is limited by diskspace and inodes - whatever runs out first.

With older file system designs (ext2, UFS, HFS+) things tend to get slow if you have many files in a directory. Usually things start getting painful around 10,000 files. With newer filesystems (ReiserFS, XFS, ZFS, UFS2) you can have millions of files in a directory without seeing general performance bottlenecks.

But having so many files in a directory is not well tested and there are lots of tools which fail that. For example, periodic system maintenance scripts may barf on it.

I happily used a directory with several million files on UFS2 and had seen no problems until I wanted to delete the directory - that took several DAYS.

It depends how many inodes the filesystem was created with. Executing

df -i 

will give you the number of free inodes. This is the practical limit of how many files a filesystem and hence a directory can hold.

  • 1
    However, many filesystems have a limit of files per directory, regardless of the number of inodes free. – Anonymous Jan 26 '09 at 7:01
  • 1
    yes, but the question targeted UNIX filesystems and as far as I am aware all modern UNIX filesystems do not limit the number of files in a directory. – klyde Jan 26 '09 at 14:33

I assume you are thinking of storing a lot of files in one place, no?

Most modern Unix files systems can put a lot of files in one directory, but operations like following paths, listing files, etc. involve a linear search through the list of files and get slow if the list grows too large.

I seem to recall hearing that a couple of thousand is too many for most practical uses. The typically solution is to break the grouping up. That is,

/some/path/to/dir/a/
/some/path/to/dir/b/
...
/some/path/to/dir/z/

and store your files in the appropriate sub-directory according to a hash of their basename. Choose a convenient hash, the first character might do for simple cases.


Cristian Ciupitu writes in the comments that XFS, and possibly other very new file-systems, use log(N) searchable structures to hold directory contents, so this constraint is greatly ameliorated.

  • 5
    Some modern filesystems, e.g. XFS don't involve a linear search. XFS's B-Tree technology enables it to go directly to the blocks and/or extents containing a file's location using sophisticated indices (from uoks.uj.edu.pl/resources/flugor/IRIX/xfs-whitepaper.html). – Cristian Ciupitu Jan 26 '09 at 2:30
  • Ah! I didn't know that. Thanks. Will add to the text. – dmckee Jan 26 '09 at 2:37
  • 1
    For ext3, you have to activate the "dir_index" feature, cf. tune2fs(8). – Torsten Marek Jan 26 '09 at 11:27

ext3 one of the most common linux filesystem formats gets really sluggish if you have around 20k + file in a directory. Regardless of how many it can hold, you should try to avoid having that many files in one directory.

From the comment you left, I think you don't really care about how many files/folders your FS can host.

You should probably consider using ModRewrite and rewriting site.com/username to site.com/?user= or something of the kind and store all your data in a database. Creating one folder per user is generally not necessary (and not a good idea).

That said, each filesystem has limits, and df can tell you how many inodes are available on each partition of your system.

Your Answer

 

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.