25

What's the maxium number of files a Unix folder can hold?

I think it will be the same as the number of files.

3
  • 3
    A much better question might be: How many should I use? stackoverflow.com/questions/466521/… Jan 26, 2009 at 1:24
  • 1
    I'd love for my site url's to look like site.com/username/ and so on, but thinking that (if im lucky) get more than 2 million users that'd be more than 2 million folders, since I don't want to use a script such as PHP with a modrewrite i was looking at the other possibilitie of folders in a folder
    – PERR0_HUNTER
    Feb 5, 2009 at 15:41
  • 1
    Do yourself a favor and create subdirectories with a rewriting scheme. Jan 21, 2010 at 12:09

6 Answers 6

21

Varies per file system, http://en.wikipedia.org/wiki/Comparison_of_file_systems

2
18

On all current Unix filesystems a directory can hold a practically unlimited number of files. Whereas "unlimited" is limited by diskspace and inodes - whatever runs out first.

With older file system designs (ext2, UFS, HFS+) things tend to get slow if you have many files in a directory. Usually things start getting painful around 10,000 files. With newer filesystems (ReiserFS, XFS, ZFS, UFS2) you can have millions of files in a directory without seeing general performance bottlenecks.

But having so many files in a directory is not well tested and there are lots of tools which fail that. For example, periodic system maintenance scripts may barf on it.

I happily used a directory with several million files on UFS2 and had seen no problems until I wanted to delete the directory - that took several DAYS.

13

It depends how many inodes the filesystem was created with. Executing

df -i 

will give you the number of free inodes. This is the practical limit of how many files a filesystem and hence a directory can hold.

2
  • 1
    However, many filesystems have a limit of files per directory, regardless of the number of inodes free.
    – Anonymous
    Jan 26, 2009 at 7:01
  • 1
    yes, but the question targeted UNIX filesystems and as far as I am aware all modern UNIX filesystems do not limit the number of files in a directory.
    – klyde
    Jan 26, 2009 at 14:33
6

I assume you are thinking of storing a lot of files in one place, no?

Most modern Unix files systems can put a lot of files in one directory, but operations like following paths, listing files, etc. involve a linear search through the list of files and get slow if the list grows too large.

I seem to recall hearing that a couple of thousand is too many for most practical uses. The typically solution is to break the grouping up. That is,

/some/path/to/dir/a/
/some/path/to/dir/b/
...
/some/path/to/dir/z/

and store your files in the appropriate sub-directory according to a hash of their basename. Choose a convenient hash, the first character might do for simple cases.


Cristian Ciupitu writes in the comments that XFS, and possibly other very new file-systems, use log(N) searchable structures to hold directory contents, so this constraint is greatly ameliorated.

3
  • 5
    Some modern filesystems, e.g. XFS don't involve a linear search. XFS's B-Tree technology enables it to go directly to the blocks and/or extents containing a file's location using sophisticated indices (from uoks.uj.edu.pl/resources/flugor/IRIX/xfs-whitepaper.html). Jan 26, 2009 at 2:30
  • Ah! I didn't know that. Thanks. Will add to the text. Jan 26, 2009 at 2:37
  • 1
    For ext3, you have to activate the "dir_index" feature, cf. tune2fs(8).
    – Torsten Marek
    Jan 26, 2009 at 11:27
0

ext3 one of the most common linux filesystem formats gets really sluggish if you have around 20k + file in a directory. Regardless of how many it can hold, you should try to avoid having that many files in one directory.

0

From the comment you left, I think you don't really care about how many files/folders your FS can host.

You should probably consider using ModRewrite and rewriting site.com/username to site.com/?user= or something of the kind and store all your data in a database. Creating one folder per user is generally not necessary (and not a good idea).

That said, each filesystem has limits, and df can tell you how many inodes are available on each partition of your system.

You must log in to answer this question.