Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

What are the appropriate options to mkfs and mount for an ext4 filesystem with a folder containing >10 million files for read access?

What I have so far:

umount /media/dirsizetest
mkfs.ext4 -L DIRSIZETEST -E lazy_itable_init=1 -E lazy_journal_init=1 -m 1 /dev/sda1
mount -t ext4 -o nodiratime /dev/sda1 /media/dirsizetest

Some context is in order. I'm doing a slightly (OK, very) crazy experiment involving seeing how different file systems perform with a single folder filled with millions of small files. Eventually I'll be filling a 1TB drive to capacity doing this (I told you it was a crazy experiment!).

The access patten is something along these lines:

Recreate the volume from scratch (using mkfs) and mount it.
Create a sub-directory, fill it with N files in sequence (named 1...N)
    (where N can be up to 2^63)
Read all files in order
Read all files in random order
Print how it takes

My natural habitat is the Windows NTFS world, and the number of options to mount and mkfs are a bit daunting. So I'm looking for guidance on what options are likely to shoot my performance in the foot.

I'm working in a 64 bit Ubuntu 12.04 desktop environment.

share|improve this question
    
you should over-think your experimental setup due to present time constraints - creating this number of files within a single directory is likely to take a very long time. Also note that ext4 would only take 2^32 files per filesystem due to the 32-bit inode numbers (you can't have more than 2^32 inodes). –  the-wabbit Jul 4 '12 at 7:15
    
2^32 files is OK if that's the file system limit (FAT32 under Windows is limited to 2^16, but I could still see a pretty clear linear O(n) slow down). And, quite strangely with ext4 and 5M files, creation takes ~0.3ms/file but random access takes ~0.7ms/file (guessing a sequential write vs many disk seeks for reads). And it's getting worse at 10M. Hence my question! –  ligos Jul 4 '12 at 22:39
add comment

2 Answers 2

up vote 2 down vote accepted

Attention: Security advice

These instructions are UNSAFE and should not be used in production environment without precautions.

For example a battery backed RAID card can help to decrease risks.

Use at own risk


If you just like it as a test environment I'd recommend the ext4 options

noatime,data=writeback,barrier=0,nobh,errors=remount-ro

This

  • disables accesstimes on read
  • writes metadata lazily
  • disables the enforcing of proper on-disk ordering of journal commits
  • tries to avoid associating buffer heads
  • remounts read only on error

for mkfs.ext4 I could only find the option

dir_index
    Use hashed b-trees to speed up lookups in large directories.

usefull

share|improve this answer
    
To anyone who happens to read the above, please understand that these options are NOT SAFE in normal production environments if you are not using storage hardware with battery-backed cache memory as found in most decent RAID cards or external storage array controllers. –  ThatGraemeGuy Jul 5 '12 at 10:00
    
Thanks for the comment! I'll add that to my answer. –  Christopher Perrin Jul 5 '12 at 10:26
    
Thanks Chris. Most of those options look like they're geared toward improving write performance, while I'm primarily interested in read performance. But they should improve the file creation phase. I noticed dir_index reading through the mkfs.ext4 man page. Wikipedia's ext4 page indicates it's enabled by default in kernal 2.6.23+, so (in theory) I'm already using it (kernal 3.2.0). –  ligos Jul 5 '12 at 22:41
    
Noatime should help with read performance because there is no write of accesstime on access –  Christopher Perrin Jul 5 '12 at 22:51
    
Yeah, I'd already picked up noatime. There was an equivalent in Windows and I wanted to replicate as much as I could, so I went looking for it. I'm waiting for my 10M file run to finish (only a couple more days to go!). Then I'll test your suggestions. –  ligos Jul 6 '12 at 6:52
show 2 more comments

Some research I did found the following links. Chris Perrin's answer provides a short list of options; these should provide additional reference material.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.