how to increase apache speed for server static files — many “R” in mod_status

Question

I have a apache serve as a proxy to backend JBoss server, I use mod_proxy_loadbalancer. I also have some static files and resources I put it in an “upload" directory which configured as a directory in apache httpd.conf. I also enabled mod_mem_cache, as I have enought memory in that server. The server is a HP DL360 4 cores / 8G RAM / with RAID card and 15K SAS Disk.

My problem is many clients reports they open static files are very slow. When I track using FireBug, I can see sometime open an static files ( html / swf < 50kb) need around 5-7s. Both the server side bandwidth and the bandwidth in my office are enough. And when I monitor the server side, the average load in apache is about 20 requests /sec.

I have open the mod_status in the apache side, I can find many requests are in "R" status which means reading requests.

What does status reading means? Is this affect the speed of server response time? How can we reduce the "R" status requests in apache configuration.

Thanks

Answer 1

A few things you can check:

• The "Reading" status simply means that Apache is reading the request from the remote client. In FireBug, or a similar tool, you can see the "request" headers being sent. These are usually pretty small and shouldn't take long to process. A lot of Apache slots stalling in the "read" state indicates some sort of issue somewhere.
• Use a benchmarker like ApacheBench (ab) to test the speed of static files. If you run it on the server you should get a very high request rate (ex: 10-20k/sec) unless something is wrong. You can also run it remotely to see the difference speeds due to the network distance and speed.
• Are you sure that the client or server network speed is not an issue? The benchmark testing should be able to confirm this and give you an upper speed bound for it. For example, if your server is just on a 10 Mbps connection it can only supply around 25 files/sec if they are 50kb in size.
• Check Apache's server-status and your server's top and iostat to check for any obvious signs of over-usage in CPU, memory or IO. Make sure Apache's MaxClients is not set too high or too low (both are bad for different reasons).
• Consider using an alternative web server for static files like lighttpd or nginx. Depending on how you have Apache setup you could save a good amount of memory/CPU by letting Apache serve the dynamic files and something else the static files. For example, on my servers lighttpd can serve static files up to 20-30k/s with only 50 MB of RAM usage while Apache can "only" do 10-20k/s with 500 MB (the memory usage is the biggest advantage).
• Check the Apache, system, and application logs for any relevant error messages.
• Another thing to check is the number of incoming requests in Apache. If you get a sudden spike in incoming requests you may get a large number of "Reads" as your server attempts to deal with them all. This could be a normal spike or could be a form of DoS (either accidental or malicious). If you see many multiple connections from the same IP in server-status, or via netstat it may be a possible cause.