Tell me more ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.
up vote -1 down vote favorite

I'm referring to file sharing websites like megaupload.com, rapidshare.com and the software download websites like download.com, filehippo.com.

They need to store tons of terabytes of data and they need such bandwidth too. I don't compare them to companies like Google, Yahoo, Amazon because, they are huge companies and they have their own personal huge hardware setup.

How do they (megaupload/filehippo) store such huge amounts of data? What technologies & hardware architecture do these use?

PS: My interest is that I want to start my own software download site like filehippo.com.

share|improve this question
why -1? Can you kindly comment? – claws Nov 26 '11 at 15:02
The downvotes are because the question is too vague and open-ended. – Ward Nov 26 '11 at 17:54

closed as off topic by Iain♦, SvW, Ward, mailq, voretaq7♦ Nov 28 '11 at 17:27

Questions on Server Fault are expected to relate to professional server, networking, or related infrastructure administration within the scope defined in the FAQ. Consider editing the question or leaving comments for improvement if you believe the question can be reworded to fit within the scope. Read more about closed questions here.

2 Answers

up vote 5 down vote accepted

Actually, what you see, is not what is in reality :)

File sharing sites dont have TONS of terabytes, but around 10-500 TB of data. You might have heard that there are many hosting companies which provide you Unlimited space, which is a BIG LIE. because there is no such things UNLIMITED space. They just have few hundred terabytes of storage space, and it is for sure that not Every user has Terabytes of data to store in webspace. Majority of users host from 100MB to 2-5GB of data/websites. the number of heavy data users is considerably low.

Coming to the point what they use? They simple have SAN (storage area networks), where every server has around 4TB to 20TB of disk space. And they are all connected through either Fiber channel or Gigabit Ethernet. At front-End, they have web application, that utilizes space from this SAN network.

As you are going to start it from scratch, for front end web application you can use any high end server that is suitable for you, and at the backend, you can build your own NAS/SAN using FreeNAS(which is Free opensource enterprise class storage management OS). you can add bunch of terabytes of cheap drives in it and your small network of few terabytes is ready. Based on traffic and usage, you can continue to add more space , based on need.

share|improve this answer
How do you know these? Where did you learn this from? – claws May 21 '12 at 10:20
1  
Based on my personal experience. as i am a system engineer. – Frank May 23 '12 at 2:12
up vote 6 down vote

Well, they have... lots of hard drives.

The Backblaze Blog discusses how they do something similar (they sell unlimited backup storage).

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.