I can recommend Building Scalable Web Sites, which is written by one of the guys at Flickr. The Art of Capacity Planning, also by Flickr bods, is also useful as additional reading. Both of those deal with the server side of things.
There's a little bit of overlap between them, but I've read the first three and I highly recommend them.
As for PHP, seems the only way to make it quick is to cache early and cache often. In particular, look at memcache and APC. Another thing to consider is that at some point the default session handler locked the session file on disk until the request completed, meaning all requests using a session would block. Implement your own session handler, but don't use a database for that.
You might find this interview interesting.