What should I know when setting up a load balanced cluster for a LAMP application?

Question

I should say, "How do I set up a load balanced cluser". I am not a sysadmin of any merit.. apt-getted and yum installed my way to some stacks in the cloud but nothing fancy.

Now I have a problem with a Drupal site that has gone beyond its hardware's capabilities. There are two servers behind a firewall, the application server which is hitting 90%+ during extra-peak times and the db/solr server which is rarely above 10% (typically 3-4) at those same times. These are managed physical hardware. DB is very read heavy.

Financially it makes sense to move these to the cloud anyway, even without a configuration change.

So what I have imagined is a load balancer running mod_proxy balancing requests between 2 (I would like to have something that magically grows) application servers synced with nfs talking to one DB server.

Currently the application server is a DELL PowerEdge 2950 MKIII with 32gb of ram, 2.5ghz x 4. Db is the same with half the ram. Both have 15000 rpm raid 1 setups.

So really, it seems doable. The load balancer upfront seems like it doesn't have to be anything at all, maybe a 1GB image?

I saw somewhere that there can be issues with cookies if the user switches servers mid-session, can anyone speak to that?

Any general advice, places to learn more, etc.. keeping in mind I really just want to knock this out rather than become an expert.

Thanks

Answer 1

Sounds like you need to add another web-server to your mix. You have plenty of DB overhead, which is nice. This sort of thing is quite doable. You already have the key points.

• Multiple web-servers.
• Data is served via NFS from a NAS somewhere, or at worst one of the web-servers themselves, and mounted by all web-servers.
• A load-balancer (mod_proxy is good, but nginx may be better) is configured to make sure incoming sessions are 'sticky' to one specific web-server.

Getting session-failover is beyond my Drupal-ken, but it may be doable. And yes, if one of your web-servers restarts for some reason those users will have to re-establish. For an application that works similar to the above, when we need to do a planned reboot we configure our load-balancer (a hardware loadbalancer, an F5 BigIP) to not allow new sessions and watch until all the existing sessions stale-out before rebooting the web-server.

Answer 2

Afaik cookies are by default stored in the DB, so they are automatically shared between the users. Just share your webroot over NFS and almost everything should work fine (some upload modules may act up if you switch mid-session).

Maybe I'm missing the point of your post, but when your Drupal server is loaded way more than it's DB, something else is wrong.

• Try enabeling drupal's built-in cache system
• Install a PHP opcode cacher (APC, eaccelerator, xcache)
• Try one of the many caching modules, if possible push full pages out to disk

I do these kinds of thing for a living; if you can't work it out: let me know.

Answer 3

Ultimately failover/load balancing must be doable on the client - while there are some good reasons for providing this functionality further along the chain (and its implicit at the network level inbetween) that invariably means round-robin DNS. Certainly for a couple of webservers using a seperate load balancer is overkill.

I'm not overly familiar with Drupal - but if the content is all stored in the database, then all you need to do is setup a webserver on the DB box and setup round-robin DNS records for the site, favouring the webserver-only box.

A quick look at the Drupal install instructions makes no reference to having writeable directories, so it looks like my assumption is correct.

In addition to spreading the load to where the capacity is available you are eliminating one of the single points of failure within your stack - the webserver - OTOH adding a load balancer means that you are adding another SPOF to your stack.