Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Are there any limitations to the amount of files that can be stored in Azure Storage?

I am concerned about storing all uploads in a single directory and wondering if they should be sharded somehow into a collection of directories.

I've searched the docs but can not find anything about this.

share|improve this question
    
How many files? –  Michael Hampton Jun 3 '13 at 3:11
    
Yeah. Bear with us, but many non professionals have funny ideas what "many files" or "large data" are. We need to make sure we at least live in the same universe. How many hundred million elements you try to store in azure? –  TomTom Jun 3 '13 at 4:14
add comment

1 Answer

Windows Azure Storage (and I'm assuming you're referring to blobs) does not have a limit to the number of objects you can store. It's only limited by the 200TB-per-storage-account cap. And you can have multiple storage accounts in your subscription (each storage account maps to a namespace such as mystorage.blob.core.windows.net).

Same goes for Table storage: no limit to the number of tables or the number of entities per table. Just the 200TB restriction.

Now, as far as directories go: Blob storage is organized by namespace.blob.core.windows.net/containername/blobname.ext. These aren't really directories. If you want true filesystem directories, you'll need to set up a disk in a blob (basically a vhd formatted as ntfs / ext3 / ext4 etc) and mounted to your OS disk. A disk is limited to 1TB (the max. size of a page blob). Once you do that, you have a complete file system to write to. Just a warning though: Only one VM can attach to a drive at a given time, so this isn't ideal if you're trying to have a file-share set up (direct blob storage is much better for that, or you'll need to set up an smb server for yourself).

One more thing about Azure storage: You don't have to worry about sharding; this is a massive-scale durable data storage system. Each blob is in its own logical partition, and the storage service organizes / reorganizes storage as necessary. This is something you don't need to worry about. The only thing I guide against is storing all objects in a single container, since enumerating blobs within a container could take a while if you have 10's of thousands of objects there. I typically don't enumerate containers, since I usually store blob uri's in another database as metadata (maybe in a SQL store, or MongoDB document store).

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.