How do you host a big data web application? [closed]

Question

Affordable alternative to Linode.

I am using linode to serve my web application. The application serves, what I consider big data using ~ 30GB for our model US State and we serve a few hundred clients each day.

We are unable to expand with linode.

Linode is very quickly becoming completely unaffordable and I would like to learn about hosting alternatives. How do you host your big data?

Should I:

• Buy my own server?
• Lease a dedicated server?
• Use a third party such as AWS?

As a hypothetical lets say, I would like to add data from every US state so 30GB * 50

What factors should I consider and what could I expect to pay using your preferred method?

Answer 1

30GB doesn't even come close to being "bigdata" (which is a wank term to begin with). 30TB or more might start to qualify as a large data set, depending on what it is and how it's accessed/manipulated.

The costs you're hitting with Linode are the same sorts of things you're going to get at any provider which has a business model that doesn't fit your requirements, with one-size-fits-all plans. It's the nature of the beast -- poor vendor selection will never begat good results.

I'm not going to try and recommend a specific provider, or a specific technology choice, because (a) we don't do shopping questions, and (b) there isn't remotely enough information to answer that.

I will, however, say that the way I hosted my biggest data set (~15TB of very precious data being accessed and updated heavily by a million or so users at random) involved a large number of physical machines with a lot of fast disks, a lot of data replication, and a quite considerable quantity of money. Cheap out on this stuff and things will not go well.