Need Suggestions / Opinions about Load Balancing + some technical questions!

Question

I have one 100TB server is this moment (the Ember model) and running few Facebook applications on it. The server is going well – no problem there with no down time.

The problem is that we are starting to expand with our Facebook applications and during the day we have like 3000 concurrent users and hopping to get more :-) As the apps go up, we soon expect to have double traffic. And I think that the server can’t support that kind of traffic as in the rush hour we get lot’s of time outs and the apps are not available. At first I was thinking that I have the server miss configured, but slowly I understand that maybe that are the limits of the server as it only has 8GB memory and raising the MaxClients in Apache will start to swap, as it is almost full now.

The option that I am thinking now is to get two Hetzner – EQ10 - i7-980X – 24 GB and one lighter Server/VPS to serve as Load Balancer to try to get better connectivity.

So I have a few questions:

1) Is the aprox 2500 users the limit of one Apache server? What exactly is the bottleneck? 2) Will in 1 to 1 comparison, i7-980X – 24 GB perform better than XEON 3220 – 8 GB, keeping in mind that one is server and another desktop processor? 3) What kind of server/VPS/Cloud I need to put in front of the two EQ10-s, so I can achieve good Load Balancing? The important is it supports all the income traffic as it’s only redirecting it, no? Say 10.000 concurrent users. I was thinking to get one VPS from Liquidweb to serve as LoadBalancer.

What do you think? My budget is about $400 - $500 per month.

Thank you all for the answers.

Answer 1

Your bottleneck is more than likely apache and its memory hunger.

A bigger server will only move your bottleneck somewhere else, likely onto the hard drives and IO speeds

If your doing load balancing then a server with decent amount of ram (probably 4gb should cut it) will be the needed, I would also recommend using a pair of LB servers in IP failover configuration which will also give you HA capabilities. for software i would recommend Pound, HA-Proxy or even nginx.

if your not going for HA capabailities but purely maximum capacity then it would be much much better to make sure your hardware is as close as possible on the network so as not to add any other extra latency which can cause other performance problems.

before you shell out extra costs you may find that switching your webserver to something more memory efficient may buy you a lot more time (and savings), Litespeed is something that can be used as a direct Apache replacement (but costs) and Nginx is free but would probably require some modification of your setup.

Other considerations would be caching, the use of Varnish etc

Answer 2

I really enjoy OpenBSD's PF when it comes to load balance.

http://www.openbsd.org/faq/pf/pools.html

It's a round-robin but you can't get any cheaper than free...

Answer 3

To answer your first question: The concurrent connection limit for Apache really depends on what you are serving. If you are just serving small static files it might be 10K while for a complex PHP script accessing the database it might be 0.1.

To scale efficiently consider three things:

1. Monitoring
2. Benchmarking
3. Optimization

Monitoring will tell you exactly what your traffic looks like and any patterns in it. A server that always gets the same traffic may be setup differently than one that gets traffic peaks x10 the average rate. It will also let you see how and where your traffic is growing.

Benchmarking will tell you what the limits of your current system and configuration are. I use ApacheBench but there are a variety of ways to test the system or just parts of it. Benchmarking and Monitoring will tell you how close to 100% usage you are and what is the limiting factor (CPU, RAM, IO, something else). Benchmarking is also needed to quantify the next step...

Optimization is getting the most out the server you already have. Make a software/hardware change and then benchmark it to see what effect it has. Even if each change only improves things 10% by making many changes it can add up to a significant amount.

Note that you can skip any or all of these items but you risk paying much more for your servers than you need to. Also, in some cases your traffic is growing so quickly you don't have time for a proper investigation...scale now and worry about optimizing it later. You can skip this step but you may

Answer 4

You are probably hitting apache's limit. It can be better with a little caching and direct serving other stuff from a lighter server like lighttpd or nginx. You can put all of this on a load balancing infrastructure as you mentioned.

Take a look at my answer on this question, I described how to do it with VPSs but it's totally applicable to your case, and maybe gives you some ideas.

Answer 5

we have like 3000 concurrent users

Not a very meaningful metric for capacity planning. Although it does have some bearing if you are running C# or Java. But you didn't say what OS / code you're running.

and hopping to get more :-)

Do you're legs get tired? ;)

...only has 8GB memory...

That's a lot of memory for a single webserver node. For most patterns of access upgrading the CPU/memory/disk in your server is not nearly as effective, and much more expensive than using multiple low spec machines. And its not just about the cost of the MIPs / disk MB/s - this also distributes load across multiple network cards. AND you get better fault tolerance.

Is there a reason you want to pay for a load balancer too? Lots of people think they need one but Round Robin DNS has a number of advantages, particulary for small cluster (and a few disadvantages - but not nearly enough to tip the balance).

Without a lot more information I can't really diagnose your problem - it could be that you've got plenty of hardware but a really bad apache config.

Have you looked at the application performance? Tuning your apache config / database schema / code can result in huge improvements in throughput.