how does a web application cope with thousands of requests?

Question

I went to a few websites and noticed that they all use AJAX technology for many tasks such as chat, messages, and so forth. They use a lot of httprequests obviously. My question is if you build a simple website using AJAX and you expected only few people per hour and then you start to have like 1,000 members logged per hour - can a single web application handle more requests per hour if you just upgrade to faster bigger servers or do you have to rewrite the code? Exactly how do you "scale" the web application?

Answer 1

That's a very, very involved question that lots of smart people spend lots of time thinking about.

That said, there are a few tried and true mechanisms you can introduce that will help with scalability. Ultimately, though, it boils down to your application and how it should specifically be scaled. For example, you would scale Oracle differently than you would scale Apache.

First and foremost, don't worry too much about what might happen. Far too many developers worry about the "perhaps" instead of worrying about the "now." The vast majority of applications and web sites do not require anything out of the ordinary.

That disclaimer aside, there are generally some principals that should be followed when designing and coding a highly scalable application. The concept of "shared nothing" comes to mind. Separation of concerns. Good developers recognize this and can build on it from day one.

A list of commonly implemented techniques:

1. Caching. Implementing both a front-end layer to keep images and data queries from hitting the backend. Also again at an object layer via something like memcached. The goal is to minimize the workload and reply to the client as far forward as possible.

2. Data store sharding. This allows the data layer to be horizontally scalable such that it's possible to add additional database systems and servers as we reach capacity. There's generally a 'shard key' or sorts that dictates which database server a given piece of data is available on. This doesn't stop at the RDBMS layer. File storage and whatnot should also be considered.

3. Load balancing. Allows us to add more front-end clients (Web servers) to handle the request load. This is where that whole "Shared nothing" approach comes in. These web servers ought to be stateless such that failure doesn't matter. It's then possible to horizontally scale by adding more machines. Since our data is spread out on a backend layer, they just drop in and handle requests. Balancing algorithms are used to pick a server which is not busy. Spread the load, so to speak.

4. CDN. This sort of builds on #1, but relies on CDN providers. We're again building on the concept of answering the request as far forward as possible. There is even a whole "Edge Side Includes" language for managing this stuff.

5. Data replication. For example, a single write node which replications to a pair of read nodes. As most web applications are read-happy, we can direct our reads to the read nodes while pumping writes through the write nodes. MySQL, Postgres, Oracle, MSSQL all support this as do the eventually-consistent NoSQL options like MongoDB. This isn't necessarily mutually exclusive to #2.

I'm sure I'm forgetting something as I'm pulling all of this out of memory, but you get the idea. At the core, the application needs to be designed to be scalable. If you have that, it's possible to spread load and grow indefinitely by deploying a smart architecture.