Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I want a cluster of 3+ VPS servers in different cities that acts like one VPS server, meaning:

% When I update a file on one VPS in the cluster, it's "automatically" updated on all the other VPSs.

% When I run a command on one VPS, it's automatically run on all others.

% When I update a db entry on one VPS, it's automatically updated on all others (which is the same as point 1 for dbs that use file storage)

% If one VPS goes down, it "catches up" when it comes back up.

In other words, the VPS are exact mirrors of each other.

I realize the IP addresses (and maybe even /etc/resolv.conf and a few other files) must be different, but the general idea is that they look identical to outside viewers.

Since the VPSs may host blogs, etc, the mirroring must be preserved when a browsing user makes a comment on a blog. In other words, I can't have a central VPS and push out updates, since users can make changes on any VPS, not just the central one.

Does such a thing exist? I tried creating my own using iwatch and MySQL replication, but it didn't work well in practice.

I know about http://mediatemple.net/webhosting/gs/ but those servers are all in a single location (and not quite what I describe above).

share|improve this question

migrated from webmasters.stackexchange.com Dec 16 '10 at 14:37

This question came from our site for pro webmasters.

1  
Is it acceptable to have a delay of several seconds between changes being synchronized? –  Jed Daniels Mar 26 '11 at 23:46
    
Yes. I'm looking for "eventual consistency" (as Amazon likes to call it). However, if someone on server X changes data and then someone on server Y changes data before the X changes are synched, it's unacceptable to lose either change (except in the very rare case that the changes conflict). –  barrycarter Mar 26 '11 at 23:48
    
Is your goal redundancy/fault tolerance? Or load balancing/scalability? –  Jed Daniels Mar 27 '11 at 0:04
1  
It's fault tolerance. If one of the servers fails, I don't want my entire site to go down. –  barrycarter Mar 27 '11 at 0:09

4 Answers 4

Anything is possible with the right technology and budget. To get an idea of what would need to be done:

1: If your using some sort of RDBMS like MySQL , you could easily setup Master-Master replication between 2+ mysql servers which will ensure all writes get to each of the other servers , while each having their own increment ID. However, High traffic databases will always lag behind each other, depending on the network layout and distance between each VPS server.

2: File sync is a little harder to do. Easiest may be to have local scripts run rsyncs every N minutes to keep all the files in sync. If these are only single VPS and not a dedicated build out in each Datacenter, then this would be one of the easiest ways.

3: Load balancing will be tough since these are only single VPS. You could do round robin DNS for each VPS ip but not the most efficient when a host is down. You could set the TTL down but some providers do not respect DNS TTL.

To give an example of what an Enterprise build out of this would be:

1: MySQL running on Primary/Secondary DRBD nodes in each Datacenter on in its own physical/virtual machine. Then each location setup for Master/Master replication. Fault Tolerant at each Datacenter, but still has Lag problem described above.

2: Dedicated SAN to store the global files that are to be shared with 24x7 Replication. Not cheap by any means but what Enterprises do.

3: Either DNS With very low TTL or BGP Load balancing device. More expensive but will be able to do better load balancing algorithms - User is put to closer region, or region is avoided when down or heavily loaded.

share|improve this answer
    
File system sync could be done using DRDB - it's a head ache, but the fun problems usually are. I'd just use the MySQL replication for that - use DRDB just for the filesystem stuff. Keep it simple ;-) –  tsykoduk Mar 31 '11 at 7:52

I am not aware of a ready made solution like this, but you can build such a cluster with Amazon EC2.

share|improve this answer
    
Details, please? I didn't know Amazon offered VPS services? I know Amazon has redundant/high availability storage, but I don't want to be at the mercy of one CPU/VPS failing. –  barrycarter Dec 16 '10 at 0:39
    
@barrycarter - Amazon's cloud service is an on-demand VPS-based cloud : they're certainly NOT spinning-up and -down physical servers on-demand! –  warren Mar 28 '11 at 13:53

It is not clear from the question what freedom there is to choose implementation technology. If you can choose a distributed nosql database like Cassandra, replicating data across many nodes should be doable, under the assumption that it doesn't matter if it takes a second or two for data to propagate.

I don't know about pushing system files, but you could probably push static web content with a clustered files system or even rsync. For maintaining the state of multiple machines, you may want to look into cfengine that will help you with both maintaining packages and configuration over many boxen.

If you're going to the trouble, you should probably do networked syslogging for these boxen as well.

share|improve this answer
    
I'll be installing apps like MediaWiki, Drupal, etc, so the redundancy has to work with pretty much any type of file. I can't limit the implementation technologies. –  barrycarter Mar 27 '11 at 0:11

Windows Server 2008 has what you're looking for built in. Just cluster the servers together using the built-in role, assign a primary node (cluster admin), adjust cluster security and settings, and then add IIS, SQL, and the others as applications and services of the cluster.

Shawn J.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.