How an online service scales high-demanding storage capacity?

Question

I was wondering how online services which have a high demand for storage capacity - scale physically? (hardware)

Let's assume a service needs to store 1000TB of data, at one geographic location, with the ability to scale. Is there a common way to approach it?

Answer 1

There are many different ways, depending on the infrastructure of the application. Network Attached Storage (NAS) solutions from companies like NetApp are common. For other applications, enterprise SAN arrays may be appropriate using iSCSI or FibreChannel.

There are even more creative folks on the hardware side, such as that detailed in this blog post from Backblaze about their Storage Pods, 67TB of NAS for less than $8k (probably even less now with falling drive prices).

Other considerations include redundancy and backup. Performance considerations may affect whether one uses enterprise arrays with SAS drives, or commodity SATA hardware.

Obviously any additional hardware comes with considerations for power, space, and connectivity. These may be handled by the company if they run their own datacenter, or they may be handled by another company via colocation or a fully managed environment.

Answer 2

There is a always room for growth when design is properly placed I would focus on 2 primary factors in consideration.

The facility at which this content resides, primary power and cooling since we have not spoke yet about the performance needs or bandwidth, this storage will need ample power and air conditioning at proper temperatures.

Secondary would be the technology solution used to store this data. An enterprise SAN device? Commodity white boxes or a combination of both and some DAS enclosures.

Answer 3

besides the neat hardware boxes mentioned by phoebus, you need some software to tie it up. One way is to run iSCSI on each box and mount them all as a big filesystem (ZFS or XFS would be happy to handle all the PetaBytes), but it quickly becomes a management nightmare and a bottleneck.

A much more scalable and failure resistant is to use a 'object store' system. There are several ones available, both proprietary and Open Source, but the ones i'm leaning to are mogileFS and GlusterFS, the first one for simplicity and the latter for maturity (and POSIX compliancy).