Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Is there an upper limit to the total number of files stored in a single subdirectory in FreeBSD 6?

The filesystem is ufs

share|improve this question
add comment

5 Answers 5

up vote 3 down vote accepted

This was discussed on the freebsd-questions list in November 2008. To quote from Erik Trulsson:

Each i-node on the disk contains a field telling how many hard-links point to that inode. This field is a (signed) 16-bit value, meaning the maximum number of hardlinks allowed is 32767. Each subdirectory created contains a hardlink ('..') to its parent, thus limiting the number of subdirectories to a single directory to less than 32767.

Note that this does not limit the number of files you can have in a single directory, since normal files do not contain hardlinks to the parent directory, but there are of course limits to the total number of files and directories you can have on a single filesystem based on how many inodes were created when the filesystem was first created.

(Full message, start of thread)

These are theoretical limits; as discussed above, you will start to run into performance problems well before you hit any limits.

share|improve this answer
    
performance issues may not necessarily be seen until too late - on relatively small partitions I've had many cases where some app will generate many millions of tiny files or link within a hierarchical directory structure that starved that partition of inodes well before space or performance were even warranting investigation. –  DisabledLeopard Aug 13 '09 at 6:59
add comment

I did some things with freebsd 6.x that involved large numbers of files (50,000+) and don't remember any specific limitations. A quick check on a 7.2 system shows the limit to be well over 100,000. The process is still running as of now, when it fails I'll let you know what the hard limit is for 7.2, which is likely similar to 6.x.

That said, you see a huge performance hit over about 30,000 directory entries when creating new files / directories. At that point people start creating files named HashOfName/name instead of just name so you make lookups easier.

share|improve this answer
add comment

I would expect that number to vary based on the type of filesystem involved as well.

share|improve this answer
    
How can I tell which filesystem is in use? –  Ian Aug 12 '09 at 20:50
    
Freebsd installs typically use ffs for their filesystems unless you're doing something exotic or mounting an alien filesystem. The exception is zfs, which is not likely to be used on 6.x. Unless stated otherwise, I would assume ffs/ufs2. –  chris Aug 12 '09 at 21:25
    
Is there a way to tell specifically? –  Ian Aug 12 '09 at 21:27
1  
type "mount" and you'll see the filesystem type and the mount options. –  chris Aug 12 '09 at 21:58
    
Added ufs to question. –  Ian Aug 13 '09 at 6:25
add comment

Your question is already answered, so just little performance tip: If you have a lot of small files you should increase vfs.ufs.dirhash_maxmem, default 2MB is too small for thousands of files.

I have such line in my /etc/sysctl.conf

vfs.ufs.dirhash_maxmem=67108864

You can read about dirhash here (UFS improvements @ BSDCON) and here (Wiki)

share|improve this answer
add comment

The total inodes are the limits of total files you can put in a directory. The total inodes are created when you format your hard drive. You can make more inode by using small size of segments. See man newfs for detail.

The vfs.ufs.dirhash_maxmem is for the memory used to hold the directory names in a directory. This only affects the perfomance, nothing limitations. If you have more memory, make it bigger, otherwise do not bother it.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.