0

I am currently using AWS infrastructure to host my mobile web application. I use Apache Web Server, mod_jk to pass on requests to Tomcat which further communicates with MySQL DB using connection pool. Of recently the server has been responding slowly. One of the cause is a request(LocationUpdate) that is hit by all users having the application randomly at 15minute interval. Which I believe is the reason for slowing the server down. And other is we are having many users as well. I was wondering how to Load Balance this.

  1. I'm not sure if its Apache who is unable to handle these many web requests.
  2. Or is Tomcat which is unable to handle them.

So not sure how should I handle it. Which config settings to confirm . I did search on internet yet wasn't sure how to figure it out. Should I increase Tomcat Heap Size ?

1 Answer 1

1

First of all, you need to reconsider, whether your app really needs to "phone home" every 15 minutes! This puts burdon not only onto your server, but also on the mobile network, on people's phones and on their data bills. In particular, if this is a location update, the app might be programmed to not send an update of the location, if the change in GPS coordinates from the last transferred update is small.

Secondly, you have to debug, what drives up the load on your web server. Use top to show load details. The line of particular interest looks like this one:

Cpu(s):  3.0%us,  6.0%sy,  1.6%ni, 85.9%id,  0.4%wa,  0.0%hi,  3.0%si,  0.0%st

This is from a lightly loaded server, that is 86% idle - so everything fine in my example. In your case, it might look like the following:

Cpu(s):  3.0%us,  6.0%sy,  1.6%ni, 4.9%id,  81.4%wa,  0.0%hi,  3.0%si,  0.0%st

The exzessive "wait" means, that your jobs are waiting for the disk to finish reading or writing data. If you have that problem, then try to reduce on unnecessary writes to disk (google for noatime to learn more about that) and try to lower write security levels on your database, where it is safe to do so. For example, if a location update is not written to disk properly in case of a power failure, the consequences to your service are likely minimal, as at the time, that power is restored after repairing the failed component, the locations will be outdated anyway. On the other hand, if new users sign up, you want to keep those writes to the main account data as synchronous, so that you don't loose accounts due to a power or server failure.

If even after those changes, wait time is high, think about faster disks: RAID 1 with two or even three disks or an SSD, before you even think about load balancing.

If your load line looks like the following:

Cpu(s):  43.0%us,  42.0%sy,  1.6%ni, 4.9%id,  5.4%wa,  0.0%hi,  3.0%si,  0.0%st

with almost all CPU time spent in userland (us) and system mode (us), then your CPU is overloaded. Check the lines below the heading in the top output to find out, which services (like apache2, tomcat, MySQL) have high cpu load. Then optimize your web application to reduce on cpu usage. If that doesn't help out, add more cpu cores to your server.

Last, but not least, check your memory with free. If a large percentage of your memory is used, or swap is used more than a few kilobytes, add more memory.

2
  • Nice answer, I would also ask why you are front ending this with Apache HTTPD, Tomcat is much faster with out it under most cases. If you need the HTTPD then you may want to try using nginx which frequently does a better job of handling multiple connections. Jan 15, 2015 at 14:51
  • Thanks Kai. I was getting 504 Gateway Error. which says because of the slow response from upstream server(which is Tomcat in this case). So I am thinking of having a separate Tomcat worker for this particular request. Do you think its the right thing to do.
    – user23577
    Jan 17, 2015 at 9:41

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .