Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I'm referring to file sharing websites like megaupload.com, rapidshare.com and the software download websites like download.com, filehippo.com.

They need to store tons of terabytes of data and they need such bandwidth too. I don't compare them to companies like Google, Yahoo, Amazon because, they are huge companies and they have their own personal huge hardware setup.

How do they (megaupload/filehippo) store such huge amounts of data? What technologies & hardware architecture do these use?

PS: My interest is that I want to start my own software download site like filehippo.com.

share|improve this question

closed as off topic by Iain, SvW, Ward, mailq, voretaq7 Nov 28 '11 at 17:27

Questions on Server Fault are expected to relate to professional server, networking, or related infrastructure administration within the scope defined by the community. Consider editing the question or leaving comments for improvement if you believe the question can be reworded to fit within the scope. Read more about reopening questions here.If this question can be reworded to fit the rules in the help center, please edit the question.

    
why -1? Can you kindly comment? –  claws Nov 26 '11 at 15:02
    
The downvotes are because the question is too vague and open-ended. –  Ward Nov 26 '11 at 17:54

2 Answers 2

up vote 4 down vote accepted

Actually, what you see, is not what is in reality :)

File sharing sites dont have TONS of terabytes, but around 10-500 TB of data. You might have heard that there are many hosting companies which provide you Unlimited space, which is a BIG LIE. because there is no such things UNLIMITED space. They just have few hundred terabytes of storage space, and it is for sure that not Every user has Terabytes of data to store in webspace. Majority of users host from 100MB to 2-5GB of data/websites. the number of heavy data users is considerably low.

Coming to the point what they use? They simple have SAN (storage area networks), where every server has around 4TB to 20TB of disk space. And they are all connected through either Fiber channel or Gigabit Ethernet. At front-End, they have web application, that utilizes space from this SAN network.

As you are going to start it from scratch, for front end web application you can use any high end server that is suitable for you, and at the backend, you can build your own NAS/SAN using FreeNAS(which is Free opensource enterprise class storage management OS). you can add bunch of terabytes of cheap drives in it and your small network of few terabytes is ready. Based on traffic and usage, you can continue to add more space , based on need.

share|improve this answer
    
How do you know these? Where did you learn this from? –  claws May 21 '12 at 10:20
1  
Based on my personal experience. as i am a system engineer. –  Farhan May 23 '12 at 2:12

Well, they have... lots of hard drives.

The Backblaze Blog discusses how they do something similar (they sell unlimited backup storage).

share|improve this answer

Not the answer you're looking for? Browse other questions tagged or ask your own question.