We're using Nginx + PHP + FastCGI for our web server and we just saw a bunch of timeout errors (see below) in the error log. We quickly restarted nginx and that fixed the problem - for now. However, we're trying to find the root-cause and fix the problem so it doesn't re-occur - ever.
Specifically,
- can you explain the error to us or point us in the right direction?
- looks like we're hitting two errors together: 18657 (connect failed) and 18656 (upstream timed out). When do they occur?
- is there a way to see the network statistics on the server when the error occurred? We'd like to see how many php processes, inbound connections, etc. we had when the problem hit us.
- how can we reproduce the problem and fix it for sure?
Here are the errors (we saw hundreds of them, posting a handful):
- error.log:2011/08/04 19:31:44 [error] 18657#0: *464859 connect() failed (110: Connection timed out) while connecting to upstream, client: 166.205.138.117, server: api.website.com, request: "POST /v3/logAgentEvent HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "api.website.com"
- error.log:2011/08/04 19:31:44 [error] 18656#0: *464476 upstream timed out (110: Connection timed out) while reading response header from upstream, client: 98.140.86.90, server: api.website.com, request: "POST /v3/logAgentEvent HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "api.website.com"