4

I have a LAMP server with a lot of hardware redundancy built in. I am not worried about the server becoming unavailable. What I am worried about, however, are potential network issues in the data center the server is in. What I would like to have is another server in another data center for redundancy. Load balancing is less of a concern.

With that said, I am relatively clueless on two points:

  1. How to have two servers in two geographically separate data centers that have exactly the same data, in terms of both files and MySQL databases.
  2. How to ensure that all traffic coming into one data center are automatically transferred to the other database in the case of a network or server failure at the first data center.

Any guidance on how to accomplish the above two problems would be greatly appreciated.

Improve this question
3
  • 1
    you may want to refer to this question and its answers: serverfault.com/questions/118096/…
    – cpbills
    May 18, 2010 at 18:56
  • I answered a similar question recently: serverfault.com/questions/141617/…. There's a lot of details here, which are influenced by budget, current platform, and requirements.
    – Warner
    May 18, 2010 at 19:10
  • The general term for this is "business continuity planning." As the linked questions/answers point out, there are a number of tradeoffs, basically between how fast you want the backup to kick in and how much you're willing to pay to achieve that. Whatever you decide, don't forget to test your plan!
    – David M
    May 18, 2010 at 19:38

2 Answers 2

Reset to default
2

Your question has many parts!

For MySQL, you will want to have two servers that replicate from each other, so that if either one is the active master, the other will get its data. You can monitor that they're getting the same data by checking how many seconds behind the master the replica's replication is. You should be writing to only the active master, or you'll probably wind up horking your database when they both use the auto_increment value.

For files, you might want to run rsync periodically to propagate changes back and forth. It might be enough just to monitor whether they have the same files - there are a variety of ways to do that.

Your second problem is more tricky. Suppose the primary data center just suddenly goes offline. How does the backup data center know what to do about it? There generally needs to be a manual component in deciding that a failure has occurred and whether to failover to the backup. To have client processes automatically move to the backup would usually require a third site that is either able to proxy connections to the active data center or to provide fast DNS switchover for names. I recommend the former, because using DNS for data center failover requires low TTL values.

Improve this answer
2
  • 3
    The third site you mention -- wouldn't that be a point of failure too? Wouldn't all incoming requests be routed through this single point of failure?
    – BrandonBT
    May 18, 2010 at 18:51
  • Well, the "third site" might itself be implemented with redundancy.
    – David M
    May 18, 2010 at 19:35
-1

For the first question, you can use mysql master-master configuration, see: http://www.neocodesoftware.com/replication/ for an example. Note the auto-increment-offset that is used to avoid conflicting primary keys in the tables between the two servers.

Use rsync to copy files between the servers, with the --delete flag to avoid having deleted files re-synced between the servers.

For the second question, I'm not so sure about this one as it's probably easier if you can do a load balancing proxy that detects if one server goes down. But this will probably create double bandwidth costs if the servers are in non-related datacenters. Otherwise perhaps you can just point the dns entry to two ip's so that the client tries them both

Improve this answer
1
  • Pointing a DNS entry to two IP address will at best, cause a client to use them in a round-robin fashion, at worst, it will choose one or the other. DNS doesn't know anything about preferring one IP over another - it maps a name to an IP address.
    – David M
    May 19, 2010 at 6:00

You must log in to answer this question.