Please tell me solution for this scenario:
- several millions files, located in one directory ("img/8898f6152a0ecd7997a68631768fb72e9ac2efe1_1.jpg")
- ~80k file size in average
- 90% random read access
- backup (replication) to other servers (every 5 minutes or immediately)
- metadata of images saves into database
When number of files have become greater than 2 millions, we got a problem with slow random access time. File system is ext3 with noatime and *dir_index* options, but there is no need to use commands like 'ls' or 'find'.
Solutions which I consider possible:
- stay with ext3 and simply convert directory tree structure to "img/889/8f6/152/a0ecd7997a68631768fb72e9ac2efe1_1.jpg"
- migrate to other file system (ReiserFS, XFS, EXT4 etc.)
- setup storage engine with distributed filesystem (give examples)
- or maybe other...
If we choose 1 or 2, how are we to replication? rsync can not handle with such a lot of datas on ext3 file system.
The best solution for us is to use Amazon S3, but this is too expensive with our traffic... Maybe you recommend some analogs (cheap CDN or open-source project)