I can recommend Building Scalable Web Sites, which is written by one of the guys at Flickr. The Art of Capacity Planning, also by Flickr bods, is also useful as additional reading. Both of those deal with the server side of things.
High Performance Web Sites and Even Faster Web Sites are taken from the Yahoo best practices for how to make your site work quicker from the users perspective. They deal more with how to get the page in front of the user quicker, rather than how to server the page faster. Things like putting javascript at the bottom of the page so the browser doesn't block. They also talk about making less requests to the servers, through things like being cache friendly and using a single image for all your icons and using css to display only a small part of that big image in the right place.
There's a little bit of overlap between them, but I've read the first three and I highly recommend them.
As for PHP, seems the only way to make it quick is to cache early and cache often. In particular, look at memcache and APC. Another thing to consider is that at some point the default session handler locked the session file on disk until the request completed, meaning all requests using a session would block. Implement your own session handler, but don't use a database for that.
You might find this interview interesting.