I'm having some issues with requests for pages not getting a response after requests take a long time to process.

I have nginx setup to use php-fpm. I have two pools setup in PHP-FPM. One pool for general web page requests, one pool to serve up image and other large files.

From my php-fpm config file:

listen = var/run/php54/php-fpm-www.sock
pm = dynamic
pm.max_children = 20
pm.start_servers = 4
pm.min_spare_servers = 4
pm.max_spare_servers = 20
pm.max_requests = 200


listen = var/run/php54/php-fpm-images.sock

pm = dynamic
pm.max_children = 5
pm.start_servers = 1
pm.min_spare_servers = 1
pm.max_spare_servers = 2
pm.max_requests = 40

Nginx is configured to use these two separate pools, with requests for images stored in Amazon S3 going through the 'www-images' pool to be resized to the requested size. From my nginx config file:

location ~* ^/proxy  {
    try_files $uri @404;
    fastcgi_pass   unix:/opt/local/var/run/php54/php-fpm-images.sock;
    include       /opt/local/etc/nginx/fastcgi.conf;

location  / {
    try_files $uri /routing.php?$args;
    fastcgi_pass   unix:/opt/local/var/run/php54/php-fpm-www.sock;
    include       /opt/local/etc/nginx/fastcgi.conf;

Because I'm testing on a terrible internet connection these requests are timing out in PHP, which is expected.

2013/01/20 15:47:34 [error] 77#0: *531 upstream timed out (60: Operation timed out) while reading response header from upstream, client:, server: example.com, request: "GET /proxy/hugeimage.png HTTP/1.1", upstream: "fastcgi://unix:/opt/local/var/run/php54/php-fpm-images.sock:", host: "example.com", referrer: "http://example.com/pictures"

What's not expected and I'd like to resolve is that any requests that should be going to the 'www' pool are timing out with nginx not getting a response from PHP-FPM.

2013/01/20 15:50:06 [error] 77#0: *532 upstream timed out (60: Operation timed out) while reading response header from upstream, client:, server: example.com, request: "GET /pictures HTTP/1.1", upstream: "fastcgi://unix:/opt/local/var/run/php54/php-fpm-www.sock:", host: "example.com"

After a couple of minutes requests to the 'www' pool start working again, without any action on my part.

I thought that using separate pools should mean that even if one pool has issues with requests taking a long time, the other pool should remain unaffected.

So my question is; how do I isolate the two pools, so that one pool being overwhelmed by requests that are timing out, doesn't affect the other pool.

To clarify, it is by design that I want to limit the number of requests that can be made at once through the 'www-images' pool. Although in practice this limit will hardly ever be reached (due to caching of the files downloaded from S3 to the server), if there is an unusual situation where that pool reaches it's limit, I want the www pool to continue functioning, as that is where the sites functionality actually sits.

  • 1. Why are images being fed to PHP? 2. Your min/max spare servers in image is rather silly. It would very frequently jump between 2 and 5 that you might even have like 1~2 process per child on avg. – Grumpy Jan 20 '13 at 8:46
  • "Why are images being fed to PHP?" Because they're stored in Amazon S3. If a request for one comes in, it gets downloaded from there to the server, and then served up as a requested size - not the original. "Your min/max spare servers in image is rather silly." I have cloudfront sitting in front of the images. It will be rare for more than 1 request to hit the server at once. The exception being on new server instances where they need to build up the cache of images. – Danack Jan 20 '13 at 9:12

I found two things:

  1. Add session_write_close(); to any long running PHP scripts as session data is locked to prevent concurrent writes only one script may operate on a session at any time.
  2. For any images that may be slow to load, make sure to load them from a different domain than the one you serve web pages and Ajax calls from, as web browsers will queue requests to the same domain when there are more than a small number of requests active.

Although these are two separate things, they had the same effect of making requests to the 'www' pool be blocked by requests to the 'www-images' pool.

Your Answer

By clicking "Post Your Answer", you acknowledge that you have read our updated terms of service, privacy policy and cookie policy, and that your continued use of the website is subject to these policies.

Not the answer you're looking for? Browse other questions tagged or ask your own question.