The one system administration reference that I go to more than any other is the Linux Administration Handbook, 2nd ed. by Nemeth, Snyder, Hein, et. al. It covers multiple Linux distros, including Red Hat, Fedora, SUSE, Debian, and Ubuntu. Even though it's focus isn't on LAMP, I believe that having a solid understanding of Linux administration allows me to use the online documentation for the other pieces in the LAMP stack--Apache, MySQL, PHP/Perl/Python--to accomplish what I need.
If you prefer books over online documentation, I've found Web Database Applications with PHP and MySQL, 2nd ed. by Williams and Lane to be a good reference for that portion of the LAMP stack.
I am neither a Rails nor a Ruby person, so I won't even pretend to have the right answer there.
Even if you aren't using Slicehost as your VPS, I'd recommend checking out their wiki and articles in addition to utilizing ServerFault.