Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

My company has a web service api which is starting to get a lot of use. Recently we had some issues with running out of memory. We optimized some inefficient code and solved the problem.

We know we are going to expand even further, we want to have a good way to deal with the intense traffic.

One idea that has come up is to have different url's for some of our heavier customers. This just jumps out as the wrong thing to do to me. The url's in some cases would point to isolated servers, but some would also just point to more virtual directories.

Is this a good solution to the problem in either case? I'm forseeing horrible maintainability issues and causing more problems then it solves. Please give me some pros and cons for both sides.

This is already on a load balanced server farm.

share|improve this question

migrated from stackoverflow.com Jun 23 '10 at 18:35

This question came from our site for professional and enthusiast programmers.

1  
How would separate URLs be any different than using a load balanced server farm? Wouldn't that just make it worse (since you wouldn't be load-balancing)? –  Brendan Long Jun 22 '10 at 19:23
    
One of my thoughts was that this would mess with the load balancing. How I was thinking of this is that the different urls would be complete instances of the api. –  aepheus Jun 22 '10 at 19:26
    
What do you mean with load and traffic? Is the CPU running at 100% –  orjan Jun 22 '10 at 19:44
    
I purely mean the number of requests coming in. CPU was high, but not 100%. –  aepheus Jun 22 '10 at 20:00

6 Answers 6

If it's already on a load balanced farm, and you're getting too much load, and you've already optimised as much as you can, the natural next step would seem to be expanding your farm to meet demand.

share|improve this answer
    
+1 Generally yes, but you can't just carry on adding extra machines without considering if there's anything else you might be doing badly. –  kaerast Jun 23 '10 at 20:58

If you are hitting the maximum capacity of the load balancer, and you have idle servers, you can try to balance more evenly using some sort of feedback, with balancers like mod_cluster. If you are still hitting the limits, you can try Round robin DNS as an alternative to handing out multiple URLs. This way you can offload the load balancing to the client. You can add feedback to this solution with lbnamed. A bigger load balancer is another approach, which of course requires more $.

share|improve this answer

Can your API take advantage of caching?

If specific parts of the API are getting called often, and returning the same results something like memcached might help you significantly.

I don't see an advantage to having specific urls for different customers. It seems to me that either you need more servers and/or your load balancing isn't working correctly.

share|improve this answer

A layer 7 load balancer would allow you to segregate those clients that are considered 'high cost' to a cluster/machine that is set up to handle their specific queries. I'm assuming that it is a 'busy' client that is doing the lions share of requests, and splitting them to their own server is why you considered giving them a separate url. With a layer 7 load balancer, linuxvirtualserver.org, you could filter particular URLs and have a rather easy to maintain system.

While you ultimately want to fix the problem the right way, using something like this might buy you enough time.

share|improve this answer

I would be looking at those heavier-usage customers right now. Can you charge them more? Is their usage more than it should be? Can you help them optimise their systems so that they don't need to be making such heavy usage? Can you add rate-limiting to your APIs? Is their usage enough to be able to provide them with their own server and charge them accordingly?

Next step is to look at your architecture. Do you have caching proxies in front of your web servers? Do you have separate database servers? Are all your servers optimised as well as your code? Are you caching absolutely everything you possibly can? Do you have good hardware?

Once you are sure your architecture is as optimised as possible you really just need to be adding extra boxes to the load-balancing setup whenever you start hitting limits (whether that's extra caches, extra web servers or extra backend servers will depend on what limits you're hitting).

share|improve this answer

Have you tried running http://developer.yahoo.com/yslow/ and http://code.google.com/speed/page-speed/ that are plugins to Firebug, trying to analyze the number of requests generated when a page is loaded?

share|improve this answer
    
We're not talking about a page here. HTML is not even returned. It generates no additional requests. –  aepheus Jun 22 '10 at 20:37

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.