@TomTom, your logic is quite wrong.
Average salary at Facebook is $98k/annum, taking into account taxes and benefits, we'll use a base cost of $117k per employee, not including office space, infrastructure, workstation, etc. Facebook uses COTS machines, lets say $2k per including switch ports, power etc. Lets assign a lifetime of 2 years to each machine, therefore, you could buy 117 machines every two years per programmer hired.
The fact that HipHip reduced computing requirements on each server doesn't add mean that their reduced hardware budget has any material effect on their ability to hire. HipHop merely helped scaling and allowed them 35% more capacity from their existing infrastructure. By Facebook's claims regarding HipHop, your claim would be that they could hire an additional 300 employees. The numbers get decidedly worse if Facebook doesn't replace servers every two years -- and we know they don't. HipHip was not developed to allow them to hire more employees, it was developed to prevent massive server requirements which add a host of other scaling problems.
Google has 470k servers based on their last public mention in May of this year. This is prior to their most recent data center opening which typically contain 80k-100k servers. Google's current scaling problem is that map/reduce jobs are constrained to a single data center, meaning that any problem they need to solve is only able to access 80k-100k servers at a time. Having this as a limitation speaks volumes about their data mining requirements.
highscalability.com, as 3molo mentions, contains articles, breakdowns and educated guesses about the infrastructure of many high volume sites.
First rule of scaling, design something that needs scaling, then fix that problem. I read a quote on the web somewhere that 'Going from one server to two servers is exponentially harder than going from two servers to three servers.'
When your application lives on a single server, you haven't hit network bottlenecks, failover issues, stonith problems, vlan and private lan stumbling blocks, worked around tcp congestion issues on your local network, writing a mysql plugin to do key/value or swapping over to udp for lossless session tracking/best effort. Until you have to start mining 750 million records a day for business intelligence or analyzing live streaming data for trends, you don't know what stumbling blocks you'll run into.
Every client interaction needs to reduce the number of clock cycles required to serve that page - this is the basic key to scaling. The fewer clock cycles required, the further your app will go on the hardware. As you start adding decisions to the page generation, you need to add other processes to continue scaling and keep your application online. Google's front end talks to a number of servers when it hands back the results of a search. Speculation is that at least 30 servers are contacted the instant you hit enter on a search query to build portions of the page which are assembled by the frontend.
While the question is quite open ended, the answer is caching. Caching is easy, cache invalidation is hard. Making sure your business process does the minimal work involved to hand back a response, making sure your datastore backends are appropriate for the problem you're solving, making sure your storage system is adequate for each step in the process goes a long way towards success.
As for some examples:
twitter: varnish, nginx, memcached, tons of servers, single mysql master that keeps track of everything as an archival purpose. Their infrastructure leverages memcached and everything is supposed to be served from ram, referring to mysql only when restoring a memcache server. Traversal of the responses is done through a list set in memcached with each timeline stored per user and updated for each tweet. (This may have been modified somewhat recently)
php.net: multiple server mirrors placed at various data centers. Their application is rather simple displaying static pages with a search function.
amazon: oracle, c++, perl (mason templating), they've recently migrated to java/servlets, etc
facebook: php, cassandra, hbase (which is used for their messaging), mysql, memcached, varnish
google: a distributed computer built on linux. If you've worked with the app engine, you understand how a sandboxed version of what google's internal systems are able to use looks like. Google invented their own system to solve problems in a unique manner. While most people use expensive computers for reliable systems, google used COTS hardware. 6 COTS machines cost the same as two really reliable computers, but, they have 3x the available CPU. This allows them to do things that are computationally expensive that other companies can't do. There is a benefit to doing things 'the cheap way'.
wordpress details their architecture on their system. Nginx, Varnish, Apache, two data centers, load balancers, etc.