Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have a server which was working ok until 3rd Oct 2013 at 10:50am when it began to intermittently return "502 Bad Gateway" errors to the client.

Approximately 4 out of 5 browser requests succeed but about 1 in 5 fail with a 502.

The nginx error log contains many hundreds of these errors;

2013/10/05 06:28:17 [error] 3111#0: *54528 recv() failed (104: Connection reset by peer) while reading response header from upstream, client: 66.249.66.75, server: www.bec-components.co.uk  request: ""GET /?_n=Fridgefreezer/Hotpoint/8591P;_i=x8078 HTTP/1.1", upstream: "fastcgi://127.0.0.1:9000", host: "www.bec-components.co.uk"

However the PHP error log does not contain any matching errors.

Is there a way to get PHP to give me more info about why it is resetting the connection?

This is nginx.conf;

user              www-data;
worker_processes  4;
error_log         /var/log/nginx/error.log;
pid               /var/run/nginx.pid;

events {
   worker_connections  1024;
}

http {
  include          /etc/nginx/mime.types;
  access_log       /var/log/nginx/access.log;

  sendfile               on;
  keepalive_timeout      30;
  tcp_nodelay            on;
  client_max_body_size   100m;

  gzip         on;
  gzip_types   text/plain application/xml text/javascript application/x-javascript text/css;
  gzip_disable "MSIE [1-6]\.(?!.*SV1)";

  include /gvol/sites/*/nginx.conf;

}

And this is the .conf for this site;

server {

  server_name   www.bec-components.co.uk bec3.uk.to bec4.uk.to bec.home;
  root          /gvol/sites/bec/www/;
  index         index.php index.html;

  location ~ \.(js|css|png|jpg|jpeg|gif|ico)$ {
    expires        2592000;   # 30 days
    log_not_found  off;
  }

  ## Trigger client to download instead of display '.xml' files.
  location ~ \.xml$ {
    add_header Content-disposition "attachment; filename=$1";
  }

   location ~ \.php$ {
      fastcgi_read_timeout  3600;
      include               /etc/nginx/fastcgi_params;
      keepalive_timeout     0;
      fastcgi_param         SCRIPT_FILENAME  $document_root$fastcgi_script_name;
      fastcgi_pass          127.0.0.1:9000;
      fastcgi_index         index.php;
   }
}

## bec-components.co.uk ##
server {
   server_name   bec-components.co.uk;
   rewrite       ^/(.*) http://www.bec-components.co.uk$1 permanent;
}
share|improve this question
    
What was changed on that day? Updated your application or PHP? What's your application? Did you enable debugging in php-fpm? –  Pothi Oct 5 '13 at 15:34
    
Nothing was changed on that day. Server config was not changed, nor were any PHP scripts. It's not out of disk space. My application is just a set of PHP scripts. I'm not using php-fpm, I'm just running php-fastcgi by doing php-cgi -b 127.0.0.1:9000. It's been working without fault for 3 years. I can't work out why it has developed this issue. –  Nigel Alderton Oct 5 '13 at 16:20
    
I had similar issue recently where nginx was complaining about Connection reset by peer while reading response header from upstream, in my case it was uWSGI which was the real problem, restarting uWSGI fixed the issue for me, as to why it was happening is a separate issue. –  APZ Jan 26 at 10:46
    
Your upstream service ( php-cgi -b 127.0.0.1:9000 ) is failing intermittently, perhaps due to increased traffic and lack of resources. –  LinuxDevOps Mar 29 at 14:53

2 Answers 2

up vote 2 down vote accepted

i'd always trust if my webservers are telling me: 502 Bad Gateway

  • what is the uptime of your fastcgi/nginx - process?
  • do you monitor network-connections?
  • can you confirm/deny a change of visitors-count around that day?

what does it mean:

  • you fastcgi-process is not accessible by nginx; either to slow or not corresponding at all. bad gateway means: nginx cannot fastcgi_pass to that defined ressource 127.0.0.1:9000; at that very specific moment.

  • your inital error-logs tells it all:

.

recv() failed 
    -> nginx failed

(104: Connection reset by peer) while reading response header from upstream, 
    -> no complete answer, or no answer at all
upstream: "fastcgi://127.0.0.1:9000", 
    -> who is he, who failed???

from my limited pov i'd suggest:

  • restart your fastcgi_process / server
  • check your access-log
  • enable debug-log
share|improve this answer
    
Ok. What are my webservers telling me? –  Nigel Alderton Mar 31 at 14:49
    
see my edit (what does it mean) –  that guy from over there Mar 31 at 15:51
1  
I see, so the Gateway in this case is the PHP server. Thank you. –  Nigel Alderton Mar 31 at 16:44

You may want to consider this git on github: https://gist.github.com/amichaelgrant/90d99d7d5d48bf8fd209

I encountered a similar situation, when I checked error logs for my upstream servers they were reporting some ulimit error so I increased that to 1000000(on both the upstream and nginx boxes) and everything worked fine

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.