We're looking for a way to improve the reliability of one of our servers (Apache/MySQL/Virtualmin setup). So far, we've had every possible kind of clusterfuck with that server in the past six months (DNS failure, DDOS, Dom0 failure, network outage, DomU failure, …; on a good day 2 at once) and while all were resolved in less than a day, it's still worrisome – there are about 50 customer websites on that host, and they will get on our throats every time the server is down (the server's availability is still over the contractually guaranteed 99%, but well… you remember the 5 occasions the server had downtimes, not 360 days it was up).
Plans so far:
- Backup DNS server (shouldn't be much of a problem)
- High-Availability setup for the server itself. The problem here is data replication to the secondary host.
The hosts would be in different (Hetzner, btw.) data centers, so we'd have a rather limited bandwidth (100 MBit uplink, and there should be at least some bandwidth left for the actual users…) and data encryption is more or less a fixed requirement.
DRBD itself scales poor over WAN, neither does it provide encryption. DRBD proxy claims to solve the bandwidth problem (but not the encryption problem, as far as I can see), but it's simply too expensive from what I read, $5k/year are too much (I'm pretty sure that's more than what we're earning with that server).
On the other hand, from my personal experience, OpenVPN/SSH tunnels are not reliable enough to guarantee we wouldn't have false alerts triggering unnecessary failovers (never mind the overhead reducing hard disk performance even more).
So... what alternatives are there? Or am I simply overlooking something?
Edit: To clarify, I'd prefer a replication on file-system/block-device level. Application-level replication is possible, but I'd rather have one replication solution running than one for each application.