Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have a custom Django app that's becoming unresponsive roughly every 5,000 requests. In the apache logs, I see see the following:

Apr 13 11:45:07 www3 apache2[27590]: **successful view render here**
...
Apr 13 11:47:11 www3 apache2[24032]: [error] server is within MinSpareThreads of MaxClients, consider raising the MaxClients setting
Apr 13 11:47:43 www3 apache2[24032]: [error] server reached MaxClients setting, consider raising the MaxClients setting
...
Apr 13 11:50:34 www3 apache2[27617]: [error] [client 10.177.0.204] Script timed out before returning headers: django.wsgi
(repeated 100 times, exactly)

I believe I am running WSGI 2.6 (/usr/lib/apache2/modules/mod_wsgi.so-2.6) with the following config:

apache config

WSGIDaemonProcess site-1 user=django group=django threads=50
WSGIProcessGroup site-1
WSGIScriptAlias / /somepath/django.wsgi

/somepath/django.wsgi

import os, sys
sys.path.append('/home/django')
os.environ['DJANGO_SETTINGS_MODULE'] = 'myapp.settings'    
import django.core.handlers.wsgi    
application = django.core.handlers.wsgi.WSGIHandler()

When this happens, I can kill the wsgi process and the server will recover.

>ps aux|grep django # process is running as user "django"
django   27590  5.3 17.4 908024 178760 ?       Sl   Apr12  76:09 /usr/sbin/apache2 -k start
>kill -9 27590

This leads me to believe that the problem is a known issue:

deadlock-timeout=sss (2.0+)

Defines the maximum number of seconds allowed to pass before the daemon process is shutdown and restarted after a potential deadlock on the Python GIL has been detected. The default is 300 seconds. This option exists to combat the problem of a daemon process freezing as the result of a rouge Python C extension module which doesn't properly release the Python GIL when entering into a blocking or long running operation.

However, I'm not sure why this condition is not clearing automatically. I do see that the script timeout occurs exactly 5 minutes after the last successful page render, so the deadlock-timeout is getting triggered. But it does not actually kill the process.

Edit: more info

  • apache version 2.2, using the worker MPM
  • wsgi version 2.8
  • SELinux NOT installed l
  • xml package being used, infrequently
  • Ubuntu 10.04
share|improve this question
1  
FYI - Some more info here groups.google.com/group/modwsgi/browse_thread/thread/… –  Chase Seibert Apr 15 '11 at 17:35

1 Answer 1

You could try to add a limit of requests after which the daemon processes are recycled (before an external process does it). This is done by adding the maximum-requests parameter of WSGIDaemonProcess.

See https://code.google.com/p/modwsgi/wiki/ConfigurationGuidelines#Defining_Process_Groups

Alternatively, you could investigate how much processes your 'django' user is allowed to have. You can check this by opening a shell as that user su - django -s /bin/bash and checking output of ulimit -a.

share|improve this answer
    
My understanding is that maximum-requests should be avoided in production code. See this thread: groups.google.com/forum/#!msg/modwsgi/KzNd5xGQjrM/i5Dz7TBePNQJ –  dbw Dec 10 at 20:25
    
Personally, I would avoid using mod-wsgi in production anyway. There are nicer options in uWSGI, such as "cheaper": uwsgi-docs.readthedocs.org/en/latest/Cheaper.html –  wkoot Dec 11 at 11:05

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.