Hardware is so cheap these days that increasingly few people even get to the point where they need more than one machine. $10,000 will buy you:
- 16-32GB of RAM;
- 2 quad core Xeons;
- RAID5 disk array.
That kind of machine can serve over 10,000 concurrent users on an even mildly optimized site for all but the most resource intensive applications.
Basically there are two approaches to scalability:
- Vertical: basically buying the biggest machine you can so you don't need more than one;
- Horizontal: doing things in a way that lends itself to parallelism. Only needed on the most intensive of applications.
Look at StackOverflow: its basically run on a Web server plus a database server and it does in excess of 6 million hits a month.
That being said, scalability is about finding and addressing your bottleneck.
- If your database is slowing things down either give it more resources or use some form of in-memory caching to take the load off;
- If disk I/O is your problem then the same applies;
- If you're running out of memory to the point where it's causing too many page faults and thus causing a disk I/O problem, add more memory;
- Does your application and it's data lend itself to partitiioning across servers? If so, that's one way of scaling horizontally;
- If bandwidth is an issue and you're delivering large files then perhaps a CDN is the answer;
- And so on.
Ultimately though, 100k hits/month is not that large. I suspect that these days on a typical Web site you'd need to get beyond 10 million/month before you had real problems, assuming you don't do things badly (eg if you don't index your database searches then of course you'll be having problems with that but they're nothing to do with hits/month).
I would say redundancy is a far bigger headache than scalability. The issues involved in having redundant links, monitoring processes for system failure, having and maintaining a DR (disaster recovery) site, dealing with the issues that entails (like split-brain clustering), etc are far more difficult and tedious.