Actually, what you see, is not what is in reality :)
File sharing sites dont have TONS of terabytes, but around 10-500 TB of data. You might have heard that there are many hosting companies which provide you Unlimited space, which is a BIG LIE. because there is no such things UNLIMITED space. They just have few hundred terabytes of storage space, and it is for sure that not Every user has Terabytes of data to store in webspace. Majority of users host from 100MB to 2-5GB of data/websites. the number of heavy data users is considerably low.
Coming to the point what they use? They simple have SAN (storage area networks), where every server has around 4TB to 20TB of disk space. And they are all connected through either Fiber channel or Gigabit Ethernet.
At front-End, they have web application, that utilizes space from this SAN network.
As you are going to start it from scratch, for front end web application you can use any high end server that is suitable for you, and at the backend, you can build your own NAS/SAN using FreeNAS(which is Free opensource enterprise class storage management OS). you can add bunch of terabytes of cheap drives in it and your small network of few terabytes is ready. Based on traffic and usage, you can continue to add more space , based on need.