Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

Cheap in terms of an individual funding it themselves, for research.

This seems like a nearly impossible task, given the size of current drives.

share|improve this question

closed as off topic by SvW, Iain, Bart De Vos, JamesRyan, womble Aug 9 '11 at 8:59

Questions on Server Fault are expected to relate to professional server, networking, or related infrastructure administration within the scope defined by the community. Consider editing the question or leaving comments for improvement if you believe the question can be reworded to fit within the scope. Read more about reopening questions here.If this question can be reworded to fit the rules in the help center, please edit the question.

3  
You really, really need to define cheap for us to give you an answer. I would define "cheap" as < $1M for storage of that size. –  Mark Henderson Aug 9 '11 at 7:59
    
I guess a better question would have been "what is the cheapest that this can be accomplished, using any means?", with the cheapest being $0 if that were to be possible. –  EmpireJones Aug 9 '11 at 8:37
    
The cheap way is to use consumer grade equipment rather than proper raid and sans. It still won't be cheap. –  JamesRyan Aug 9 '11 at 8:53
2  
Blazeback backup created their own custom 4U servers that store 135TB each for $7,867 (Or $0.0569/GB - $59,672/PB, again that depends on your idea of cheap). I'd guess disk IO is terrible but depending on your appication it might still be a viable solution (I.E. perminant storage/archiving - Write once and forget forever). They claim to be able to saturate a 1GB network and the disk read/writes keep up blog.backblaze.com/2011/07/20/… –  sam Aug 9 '11 at 9:29
    
*That doesn't include power, bandwidth etc. that's the raw hardware cost –  sam Aug 9 '11 at 10:10

6 Answers 6

Although SvenW's is right in that you'd need ~340 disks (each drawing around 15 watts of power btw) and thus the raw disk cost, the hard/complex bit would be the controller and cabling management. Even if you used those 24/25 disk eSATA enclosures/extenders (which need power too), you'd still need 14 of those, most disk controllers have no more than 2 eSATA connections each so unless you wanted to daisy-chain them (bad idea) you'd need 7 of those cards (each drawing power too). So you'd end up with a machine with a lot of cards, some on wider buses than others, oceans of cabling and piles of power-hungry, hot individual disk boxes. And designing some form of RAID protection into the whole thing could get seriously complex, especially given you'd want to design the thing to survive a single disk, single enclosure and single controller failure.

I do big storage and as far as I'm aware big and cheap equals long term headaches.

edit - out of interest I ran some numbers;

enter image description here

Oh and I found a card that could in, theory anyway, connect to that many enclosures, not found out yet if it'd support that many disk though.

Oh and that controller can apparently deal with that many enclosures and disks - who knew! would be a lot of contention though.

Hope that helps.

share|improve this answer
    
I didn't even want to go there for this question, because 340 disks equals non-cheap for nearly everyone, even if everything beyond that point is the part where the fun begins.. –  SvW Aug 9 '11 at 8:31

Define "cheap". Backblaze have a famous blog post outlining their pod system for massive storage, but be aware that IO access will be slow.

You would need 16 backblaze pods to reach 1Pb of raw capacity, more like 20 or so to reach 1Pb usable capacity

share|improve this answer
    
I recalled the backblaze pod myself, but couldn't find the article. Truly an impressive piece of work. –  MadHatter Aug 9 '11 at 8:33

Think about this: You will pay at the very least $20000 for the disks alone (without any redundancy, 340 x 3 TB x $60), without any cases, servers or controllers.

share|improve this answer
2  
I have to add, given that we have no idea where the OP lives, the price of a 3Tb drive would vary wildly. I wouldn't expect to see one for < $90-$100 where I live. The difference adds up when you're talking about hundreds of units. –  Mark Henderson Aug 9 '11 at 7:59
    
> United States –  EmpireJones Aug 9 '11 at 8:38
    
You can get 3TB disks for $60?! The cheapest I found was ~$160. –  Chris S Aug 10 '11 at 2:58

No problem if you want to experiment with a new way of storage. Look at Ceph which is designed for that.

If you now say "No, that's not what I'm looking for" then please specify your needs except "cheap".

share|improve this answer

The only way to do it right is in a SAN. Unfortunately they aren't what you call 'cheap'.

share|improve this answer

If the goal is 1PB of storage, then the above numbers give you a lower bound of something under $100k to something on the order of $1mil. Anything that involves 400 give or take spinning disks, controllers and cases to hold such disks, etc, will not be cheap either to run, cool, or maintain since there is close to a 100% chance than at least something will be broken today :-) This is unless you go with some solution designed to do 1PB of usable storage but requirement #1 for those sorts of solutions is "buy or lease some computer room."

Keep in mind as well, it takes just a little while to write 1PB of data. It takes on the order of 3 hrs to write 1TB of data, give or take, so about 125 days if you don't do it in parallel. Reading is on the order of just as fast. So, backup is a bit slow too.

If what you want to do is experiment with a distributed file system, then, do it with fewer and smaller disks. Pick up some oldish computers off of ebay, stuff some 1TB (or what ever is the cheapest $/GB disks) in them, and start researching.

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.