Overloaded webserver issues

Question

I am new to the area of web development and currently interviewing companies, the most favorite questions among what people ask is:

How do you scale your webserver if it starts hitting a million requests?

What would you do if you have just one database instance running at that time? how do you manage that?

These questions are really interesting and I would like to learn about them. Please pour in your suggestions/ practices(that you follow) for such scenarios

Thank you

edited - This is a general query and can have multiple bottlenecks. I am interested in learning about them. Thank you

Answer 1

Oh boy, I wonder if a question could be more generic. But this is such an interesting topic I'll share my thoughts:

Bottlenecks for static content

Serving out static content is relatively easy. The first bottleneck you'll see is most likely your network bandwidth. Disk I/O might be another bottleneck if you have lots and lots of content to serve.

Misconfiguring can lead to bottlenecks with static content, too. One way to see some serious trouble with static content is if you have a single Apache instance serving out dynamic content, static content (small objects, such as navigation graphics) and large static content (video streams etc). That way if there are lots of clients downloading video streams, they will occupy Apache processes for the duration of the download. Typically that will eat up lot more resources - especially memory - than a separate Apache with mod_php or whatever modules you need for dynamic content, and a separate, more lightly configured Apache (or, preferably lighttpd or nginx) for static content.

Bottlenecks for dynamic content

Dynamic content is a different beast and the source of slowness can be very tedious to track down. There are some typical bottlenecks around, though:

• Bloated code. One PHP framework I saw loaded dozens of very heavy classes and initialized all kind of caches and whatnot during every page load. Bloat leads to slowness. Slowness leads to suffering. A simple "hello world" page with that framework brought an 8-core server down to it's knees -- Apache benchmark, Siege and JMeter were able to serve less than 10 requests per second. Around a request per second per core is jusssssst a little bit less than I'd expect ....

• Too many SQL queries. Even simple queries do stack up, if there are enough of them. If your front page generates 100 SQL queries within each page load, that will be a trouble for you some day.

• Bloated SQL queries. If your queries typically contain 17 JOINs, UNIONs and all that jazz, you can be sure that something could be optimized somewhere.

• Lack of caches. If your content doesn't change too often (your site is a news site, for example), it makes sense to cache things and not fetch everything from the database with each page load. Also see memcached.

• Bloated database. Sometimes it might make sense to archive items from the active database table to some archive table and only use that archive table during search or other operations requiring access to all data. A smaller table for active content means that database can cache everything in it to RAM.

• Lack of database indexes. Full table scan is not a good thing.

• Too slow server. Sometimes your server (farm) just might be too slow for the current task. Identify if it's because of lack of RAM, lack of CPU horsepower or something else, and upgrade accordingly.

I hope this answers at least some of your questions. :)

Answer 2

This should be a wiki, and the question is unclear (queries suggests the database, but webserver suggests you mean page requests).

Anyway, here's what I'd do in general, but it doesn't apply for all situations:
Internet -> Redundant Firewall Pair -> Squid Proxy -> Redundant Loadbalancer Pair -> Multiple Webservers -> Multiple databases in a master-slave MySQL replication setup.

All database writes going to the master (multiple masters if a lot of writes) which is propagated to multiple slaves. Database reads/queries are sent to the slaves, and only the slaves.

BTW, 1M hits is a very inaccurate measurement of traffic. 1M static content requests could be better than 1k requests to a PHP file which causes database lockage for a couple of seconds each time.

Answer 3

Where exactly is the bottleneck? HTTP requests, NFS, so much to talk about.

How is the disk connected to the database server? SAN, NFS mounted directory, etc...

Answer 4

The short answer to the first one is probably load balancing and reverse proxying, and to the second one memcached. Obviously you want to add on to that, motivate, explain, etc.