I've been working on it for a few weeks. I've an IIS 8.0 Server hosting, ASP .NET 4.0 application in Classic mode. Two of the files are GET which do file system reads for every request and one is post which writes to file system. Now, I sometimes start getting "Server too Busy" from a server. That error is intermittent when the server reaches this state, i.e., some of the requests return 200 while most of them return "Server Too Busy". And, even restart of IIS server doesn't make it behave normally.
I read regarding the possible causes of this issue. Basically, Queue size exceeding a limit, timeout of requests in queue, etc. should be the reason. Now, there are queues are various levels, HTTP.sys for Application pool, ASP .Net global queue, Application (AppDomain) queue, CLR queue.
I tried to reproduce the error by sending lots of GET requests to the server simultaneously, but none of the queue size seems to be increasing much as per perfmon. I started getting "Connection Timeout" due to this, but not "Server Too Busy". Where am I going wrong in reproducing this? Is it because I am sending the same request every time and it is getting cached by http.sys cache itself?
To make sure, I am able to figure it out the next time I encounter it, I want to log various queue sizes periodically, number of requests pending on the receiver, the number of requests for each of the API currently in process, i.e., log any of the parameters that may cause this problem periodically. How can I do that?
I didn't get any error log in ASP .NET logs. In HTTPERR, I can find many "Timer_ConnectionIdle" errors and very few "503 N/A" (Service Not Available).
Please help. Thanks in advance.
Edit: On Tracing the "Server Busy" request, I got a warning events. Here is the trace of that:
-MODULE_SET_RESPONSE_ERROR_STATUS ModuleName IsapiModule Notification EXECUTE_REQUEST_HANDLER HttpStatus 503 HttpReason Server Too Busy HttpSubStatus 0 ErrorCode The operation completed successfully. (0x0)
Can anybody please provide any insight into it?