0

I am working on a website where users have ability to store all kinds of media on the server and there can be any number of users.

I thought I would store all the media which users upload like this:

MediaRootDIR/UserRootDIR/file.xyz (UserRootDIR is unique for every user)

There is another way of storing this:

MediaRootDIR/file.xyz

I dont worry about organization cause I have path of the files stored in DB anyway.

My Question:

Which method is faster? (Search,Retrieve & Write for Filesystem - Using EXT4 File system as of now).

Does using subdirectories speed up the retrieval since less no. of files need to be loaded every level in the directory tree?

Is it advisable to use subdirectories since I heard there is a limit to the number of subfolders a folder can have in EXT4 is around 64,000

Is there any better alternative to store media like this?

Thanks in advance

EDIT: Name conflicts wont be a problem for me since every file uploaded has unique name

1 Answer 1

1

EXT4 (and even recent EXT3 with dirindex mount option) and XFS have no problem storing millions file per directory. However, I strongly suggest you to use the first approach (with the UserRootDir intermediate directory), as it is much more manageable. For example, think about issuing a simple ls inside a directory with 1 million vs 10.000 files: the former will retrieve 100 times more data than the latter.

3
  • Doig anything n a large directory becomes an exercise in patience and pain. Making them smaller is a good thing.
    – user9517
    Nov 13, 2015 at 22:31
  • Thank you. So, will it have any impact in data retrieval speed? Nov 14, 2015 at 9:47
  • With modern filesystems, speed on large directories is not a problem. The main difficulty stems from managing such very large directories.
    – shodanshok
    Nov 14, 2015 at 10:12

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.