What's the most appropriate way to constantly mirror all the settings and data from one Linux machine to another? I have a critical server, which I'd like to have a "hot spare" available for in case any component failure takes it offline. I have RAID and redundant power supplies, but a failure in memory/CPU/motherboard could still take it offline. Is using rsync on a hourly/daily basis to copy over the filesystem to an identical machine enough?
There isn't "the" way to do this, but a whole pile of ways that depend on your needs. rsync can do enough copying that recovery is possible, but isn't necessarily the answer you need.
Is using rsync on a hourly/daily basis to copy over the filesystem to an identical machine enough?
Good question, is it? There are two issues that will present:
- You can lose up to one hour, or one day, of changes to the system and your data. Is that acceptable to your business?
- rsync takes time to run, so if the whole process takes five minutes you will have some files from 12PM, and some from 12:05PM. Is that acceptable to you?
- If you lose the source machine during the rsync process you might have a broken clone. Is that acceptable to you?
You should also watch out that, for example, database servers don't really cope well with nothing but rsync clone backups unless you take extra care - they are very sensitive about lots of different files matching, and if they are running at the time you rsync, you can have problems.
You can improve on point 2 using LVM snapshots, which will give a consistent instant at which you capture the machine - that makes your clone the equivalent of having a hard reboot when you start it up, which is not too bad to recover from.
You can try and configure rsync to minimize the risk that 3 bites, but you can't totally eliminate it.
Ultimately, though, it is all about what your business needs and the trade-off. Good disaster recovery backups help mitigate the problem.
Pacemaker would manage machine-level resources such as the networking failover (moving IPs between systems), starting and stopping applications (apache, mysql, postgres, etc).
DRBD would be used to do the on-disk block-level replication to help ensure that your data is as up to date as reasonably possible between your primary machine and hot spare.
There is no one solution for backups.
For non DB data, I agree with Travis. Pacemaker/heartbeat & DRBD are the way to go.
For settings, it depends how fancy you want to get:
- version control
- configuration management tools (puppet, cfengine, etc)
- ssh + ....
take your pick.
Databases are another beast. Copying their files may work, but I don't recommend it. Most db engines have some form of replication built in these days. See if your's does and if it meets your needs.