I am running a API server on an EC2 C1.medium instance. Our API call takes around 3-5 seconds to finish (runs a python call and uses NLTK calls which are little memory consuming). We have configured it atop an Apache server and when we try ab to benchmark, we can just about make 100 concurrent calls. If push this above 100, we get apr_poll: The timeout specified has expired (70007) or apr_socket_recv: Connection reset by peer (104) When the checked the system load, Apache usually crashes with out of memory, but the CPU load is also around 90-100 all the time. Is this the absolute limit of EC2 C1.medium instance, should we move to a bigger instance or is it because of improper tuning of Apache?
Before upgrading instance type, First try to find out that which resource is consuming so much system resources. For example, in your case, the TCP/IP kernel parameters can be tuned to withstand high level of bandwidth. Also, disk can optimized as well as apache can tuned for optimal performance. But before doing all of this, try to look at your application, that it actually does require that level of resources or not. then go for other steps, one by one. and in the END, take step to upgrade instance type from medium to Large. By doing this analysis, you will also come to know that your application is CPU intensive or Memory intensive, so you can choose High cpu or High memory instance, based on your demand.