We are currently moving our single Linux server to a few other physical Linux servers, and would like to take the opportunity to have some more failover options.
Backups are fine, and we don't "really" need HA, however, if we are looking at ~1 day for replacing hardware/restoring from backups, it's starting to get a bit too much. Downtime of an hour or three is "okay" if there are any really critical hardware failure (if nothing else for waiting for DNS changes).
We are currently running a bunch of KVM-virtualized images for apache (several hostnames), mysql and exim on a single machine. That machine has several external IPs (one per virtual image) which I think would be one of the bigger issues if there was a problem. Our hosting (Hetzner) would need to be contacted to re-configure those IPs to a different server in case of hardware failure (we currently have a second machine which does snapshots of the virtual machine images, and also file-system based rsync-backups of the virtual machines).
We are planning on running apache + mysql + exim on say 2-3 available physical machines (excluding backups).
How should I set it up to minimize downtime when there will be a hardware failure?
Still virtualized? DRDB for storage? nginx proxy/HAproxy for the web server? Not using a bunch of external IPs for various services? HA? Failover? Redundancy for performance?