up vote 3 down vote favorite
1
share [g+] share [fb]

I'm trying to get an overview of how i would go about creating a load balanced web server setup. Setting up the actual load balancer and adding two or more web servers seems fair enough. However, I can't decide what the best setup for files would be.

The web servers will be running a CMS system (in this case Drupal). There might be content uploaded by users which relates to content, and this should be accessible to all web servers. Now, I was thinking of simply having a designated file server and mount a file directory on each web server so that the various web servers could access these "shared" files.

However, would this solution be a bottleneck if we're talking about 4-5 servers? I'd think the mounting would be done over SSH.

All help highly appreciated!

link|improve this question
70% accept rate
feedback

4 Answers

up vote 4 down vote accepted

Before I moved to the CDN I had a web farm of about 20 servers. I started with NFS until I hit a bug in the linux kernel about 3 years back that caused an issue with not mounting the volume and it wasn't failing over correctly.

I then created an rsync script that would sync the dir's every 5 minutes to each image server. This wasn't ideal but it worked and I didn't have any issues with it.

The next step I switched to lsyncd which uses inotify to monitor a directory and when a file is changed it spawns a process to sync that file. By default that's rsync. That solution was solid and if not for the economy of a CDN I would have continued with that.

Lsyncd: http://code.google.com/p/lsyncd/

Hope that helps.

link|improve this answer
This is interesting...have you replaced the "public" and "private" file systems altogether with something like the Storage API? – sbrattla Oct 7 '11 at 5:31
I'll mark this as the right answer, as it gives the most "hands on" description based on practical experience. Thanks! – sbrattla Oct 7 '11 at 5:46
feedback
up vote 7 down vote

John and Cailin's five-step tutorial on scaling Drupal will be very helpful to you.

The first two steps are as follows:

Step 1: Separate your web server from your database server.

Step 2: Set up redundant web servers and load balancing.

Bottom line: you are not going to want to scale horizontally (adding multiple servers with identical functions) without first scaling vertically (separating functions that currently reside on the same server into different tiers).

link|improve this answer
Thanks! It seems that the tutorial only describes a single db/file server. If i went a little further and separated the db and files, would you have an idea at what point a single file server might be to little and i would have to look at balancing the file servers? – sbrattla Oct 6 '11 at 6:47
According to J and C, not soon. I have added a link to the top level of the tutorial so that it is easier to explore their full recommendations. – Miles Erickson Oct 6 '11 at 15:42
Thanks for your answers! I have had a look at the links and they give a good insight! Thanks for sharing. – sbrattla Oct 7 '11 at 6:10
feedback
up vote 4 down vote

I have used glusterfs (http://www.gluster.org/) to share files between load balanced web servers. I tried to also share the MySQL database files between the servers as well with glusterfs, but with no success - I recommend to separate the web from the database server as the first poster suggests.

Other solutions to share files between servers are lustre (http://wiki.lustre.org/index.php/Main_Page) and, depending on the load balancer setup, you could also use lsyncd (http://code.google.com/p/lsyncd/) to synchronize files between the servers.

link|improve this answer
Thanks, lsyncd seems easy to get up and running. I'll have a look at that too... – sbrattla Oct 7 '11 at 5:44
feedback
up vote 0 down vote

You can set a highly available NFS share and mount it on every http server to serve your files.

Please find a tutorial for a highly available MySQL cluster: http://www.howtoforge.com/loadbalanced_mysql_cluster_debian

link|improve this answer
feedback

Your Answer

 
or
required, but never shown
discard

Not the answer you're looking for? Browse other questions tagged or ask your own question.