How many server side resoures I’d need to build a website of over 200 million pageviews per month, each page has a size of around 400kb, could anybody help me to figure it out or estimate it?
How many servers i need? Which is a good provider, i know some but i just want to see different options
I'd highly recommend picking up:
Scaling to millions of visitors per day is not a trivial subject and will require some serious planning and load testing. Wikipedia, for example, runs a cage of about 350 servers or so at a co-location facility in Tampa, FL (I've seen it; I have servers there as well) which runs one of the most highly visited sites in the world. Their architecture is very different from a site like Facebook which has been estimated to be running some 60,000 servers at various data centers around the world.
200 million page views on a page that's almost half a meg in size, connects to fifteen different databases, and the application is written on Rails? Woof.
The first issue you're going to run into is that a 400kb page is obscene. Serverfault's main page, with it set to display 50 questions per page, is 19k. CNN.com runs a whopping 21k, images and all. To even consider running 200 million pageviews a day, you're going to have to get that page under control. 200 million x 400k = 74.5 terabytes. PER DAY. Most places who host, host in terms of terabytes per month, not petabytes per month. You'd be moving into Google territory, only with about 1% of the pageviews.
The second issue you're going to run into is that Rail's isn't easy to program in a way that makes sites scale in a linear manner. I won't go so far as to say that it doesn't scale, but many many people have had trouble with it, and you're going to need to pay extra attention there.
The third issue is going to be your databases. 15? Really? That can't be allowed to stand. You're going to need to look into de-normalization, start working out how to build star tables from the disparate databases, and trim that number down. Databases are the places where Rails has it's problems and you're going to want to minimize the number of connection objects you track at any given time.
Until you've got that stuff in hand, there is no point in looking at hardware or hosting. You can throw as much hardware at it as you like.
It sounds like you're still starting up. In reality there are so many variables it's next to impossible to tell in advance what hardware you will need.
It's probably best to start fairly small, be flexible, measure all the metrics, react fast and keep a close knit between devs and operations.
Be prepared to re-architect, maybe serveral times. 200M pageviews/day is something, but in a well designed environment it's very doable, even on a budget.
As for the provider, try to find one that has some reputation and matches your own scope/specialties. Decide if you want to do network, hardware replacements, OS deployment, operations, monitoring, or do you want to leave it to the provider. Also, don't go with the cheapest.
If you have more specific questions, shoot.
This is going to sound harsh, perhaps it is, but for this anticipated level of usage you need to employ a-person/people to design your scalability.
For a start you need to ensure that you're caching the right data types in the right places in the chain. You need to ensure it's secure, resilient and supportable. You need to consider your SAN setup, failure scenarios, traffic flows, bandwidth pinch points, serviceability strategy, backup and DR systems - and their the things I can think of from the top of my head.
Also you need to calculate whether to do this with dedicated kit or shared, shared could easily work out more expensive with this load.
Basically if you're planning on building something this size yet asking this type of question, no insult meant, but you need extra help.
OK for more information, the application is written in Rails, there are like 15 different big databases involved to get the data from page, the page is a mix of text, images and different applications made on flash.
I just need some clues to estimate the servers i need, i know i will need a small server in the begining but i expect a big growth rate, so can you give me some exaples of how many servers certain pages uses, like for example espn.com or cnnsi.com, facebook, or something like that.
