stable, recent, free single system image solution for linux

Question

I've started looking into creating a load balanced virtual server, for running mostly web services, project management services (version control, etc..), and applications of that sort. And I need an open source (Linux) solution.

Wikipedia has this entry, there are seemingly very promising stable projects, but most are long dead.LVS and Kerrighed look possible, but I am not sure. Are they worth investing (timewise)?

What would be a good solution? (although I can not afford a commercial solution (Linux or otherwise), I would like to learn about these alternatives, and appreciate comments to that end).

thx

Answer 1

It looks like you are trying to solve the problem at the wrong layer. I don't know any sane system administration that would try to use Single System Image to run a web server when there are other methods such as reverse proxies that are much simpler and as a result more reliable.

Such As:

• Pound
• Varnish
• NGinx
• Squid

Answer 2

Unless I'm really reading the question wrong, I'd say for hosting web applications, you're going about it the wrong way.

I'd suggest having multiple nodes (virtual, or physical) and managing their config with puppet.

Your nodes could be a whole rack of 1U servers, or a bunch of powerful 3U multiprocessor servers running KVM and then an OS of your choice as virtualization guests.

Given 4 servers you could set them up as follows:

• Server 1: Load balancer + HTTP Node (running Varnish and Apache)
• Server 2: Load balancer + HTTP Node (running Varnish and Apache)
• Server 3: HTTP Node + DB Master(running Apache and MySQL)
• Server 4: HTTP Node + DB Slave(running Apache and MySQL)

It would be advantageous to have a fifth server, that runs services, such as nagios, munin, tftpd for a PXE boot environment, a small HTTP server for kickstart/preseed files, a DHCPd, maybe serial consoles via a Rocketport or similar.

The massive advantage of using Puppet to deploy your own systems, instead of having a single image, is that the resources are effectively self-documenting. It's a lot clearer, and less black-box than just having an image you drop onto servers. Plus it makes updates and changes to the image far simpler.

Answer 3

I'm not sure I'm answering what you're actually asking, but if you're looking for a way to take a VM and mirror it, you can use any of the free virtualization tools I'm aware of (VMware Server, ESXi, kvm, etc)

• make your VM with everything you need on it
• copy the VM
• make changes to the copy (ip address and hostname)
• start both VMs
• insert load-balancer (hardware or software, doesn't matter)
• .. no 6th step I can think of :)

Answer 4

As exciting as SSI sounds, they're quite unlikely to perform optimally.

Since your main target are web apps, you can (should!) use the current best practices. Typically, these start by:

• a caching load balancer as frontend (squid, varnish, nginx)
• several HTTP servers for web apps (typically apache, might be nginx+FastCGI, whatever)
• database

if well done, your first bottleneck would be the database, at this point, you should:

• add cache to your web apps, to reduce DB hits to a minimum. (modern frameworks (RoR, Django), include great support for memcached)
• take some kind of jobs from the DB to more specialized apps. first candidates are task queues (to rabbitMQ or similar) and key/value stores (to tokyo cabinet, resis, mongoDB, lots of them)
• distribute the DB. if it's many reads/few writes, try master/slave replication (easy on MySQL), but if this is your case, memcached should've absorbed most of the load already. Also try sharding.

if you ever get to overgrow this (are you FaceBook?), you'll have to rethink your whole structure, à la Google (where they do almost everything 'off the line' with MapReduce).