Amazon, Facebook and Google must take in TBs if not PBs per day, so does that mean they have people continuously installing new hard disks and cables in new racks in the same way you might build a regular server, or is there a different technology used for connecting drives en masse?
2
-
2en.wikipedia.org/wiki/I,_Pencil– Michael HamptonAug 30, 2013 at 6:51
-
Another facet to think of is drive failure rates with that much storage and how they keep up. Some people have estimated Google might have a new drive failure somewhere every minute.– MattSep 26, 2013 at 14:04
Add a comment
|
5 Answers
I don't know if any of them are actually installing hardware one server at a time. Back in 2008 MS started building its data centers by getting servers delivered in sealed and pre-wired shipping containers of servers that they just needed to unload from a truck and plug power/network connections into. While the 08 build was a mix of containers and traditional for their most recent datacenter they've since gone to a custom prefab design that's weatherproof and doesn't need to be housed inside separate buildings.
Both HP and IBM sell similar packages with prebuilt containers full of servers that just need power/data connections to deploy.
answered Aug 30, 2013 at 13:39
-
This is, to me, the only answer really addressing the initial question. +1– mverooneAug 30, 2013 at 14:00
-
1BTW, here's the source for that picture. It's a Microsoft Azure datacenter in Virginia. Sep 4, 2013 at 21:56
-
Someone still has to build/connect/test the shipping container server at a time and google/facebook tend to do their hardware in house.– MattSep 26, 2013 at 14:04
-
@mindthemonkey that's done in the same factory using the same cheap Chinese labor that ordinary rack mounted servers/blades used by the rest of us are assembled in. Sep 26, 2013 at 14:23
-
Google has several technologies which they developed internally to store these huge masses of data. Using these technologies they can actually add truck loads of hard disks into their cluster without any downtime, but yes, they do still need people doing that.
As far as I know from the Google Blog the two main parts are the Google File System, which is a distributed file system that can scale up to really big scale: Google File System
And on top of the Google File System they have Big Table which is some kind of Key Value database and also scales up into huge scales: Big Table
For guaranteeing high availability everything is redundant many times, more than 3 times in most cases.
-
1I think the question is more hardware-oriented, and your answer a bit off-topic, but this is a good-to-know information.– mverooneAug 30, 2013 at 13:58
That's precisely correct. I remember that at one time, Facebook datacenters were adding three tractor-trailers full of hard drives and rack-mount servers in the average day. Of course, they have complicated schemes to make storage scalable and redundant. Google, for example, has GFS. Facebook has three data centers just for their equipment, each larger than two Wal-Marts and a new one planned four times larger than their existing centers.
answered Aug 30, 2013 at 6:51
With new-generation Open Compute storage solutions able to fit 180 TB of disk in 4 rack units, a PB is not really a huge amount of space: adding 1PB a day would mean racking 5 such servers a day, not a big problem. Better yet, these can be had for about $10K, which means you are paying less than $60 per TB.
So yeah, the technology is available to all of us to do the same, at a price that doesn't break the bank.
But you have to keep in mind that larger companies always get a better deal, and sign large contracts to procure vast amounts of storage. They may get small shipments spread out over the year, but it's not like they order just 1PB at a time.
-
1The open vault specs are online. With 4TB drives they are up to 120TB in 2U.– MattSep 26, 2013 at 14:03
Here is an interesting blog post from BackBlaze on how they do this (they proably don't need as much disks as Google or Facebook, but still a whole lt), and what they needed to do when there were floods in Thailand that made harddisk expensive and harder to get by:
http://blog.backblaze.com/2012/10/09/backblaze_drive_farming/
