With millions of users searching for so many things on google, yahoo and so on. How can the server handle so many concurrent searches? I have no clue as to how they made it so scalable. Any insight into their architecture would be welcomed.
3 Answers
There are numerous case studies and talks by Google engineers available online with a little searching. Suffice it to say that Google Search is highly distributed and pushed out datacenters all over the world.
There's a ton of information available over at http://highscalability.com/google-architecture.
-
One example is the paper published years ago on disk failures (at the time, estimates were they had 800,000 servers). And the paper on MapReduce, just to name a two. Oct 9, 2013 at 2:32
As has been mentioned, the networks and architecture of large-scale websites is highly distributed across many data centers and tens of thousands of servers. If you're interested in how this works, I'd recommend a book called Scalable Internet Architectures which describes some of the concepts and theories behind scalable and distributed systems.
Among other things, they use 100,000's of servers, and MapReduce. Massively parallel.