Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

My site on Webfaction is using the Requests python package to make a good number of requests to another site. This happens whenever a user triggers a particular ajax request on mine. (I on-demand scrape the other site to provide the info, and it involves navigating through around 8 pages via post requests, and two or three parsings via the BeautifulSoup package.)

When I'm making normal requests to my site, the apache worker memory usage is very stable at 33MB or so. However, each time I make a request triggering the actions to get this extra info, it leaks about 12MB (!).

I'm thinking this may have something to do with making the requests out, but I really am no expert at the workings of Apache, mod_wsgi, or Django. Any insight would be appreciated! :)

P.S. Merry Christmas! :)

share|improve this question
    
Merry Christmas to you as well! –  EEAA Dec 25 '12 at 7:16
1  
I put this in comment, because it's not a direct answer but a cop-out solution. If you set a low value of MaxRequestsPerChild in apache settings, it should kill the child and reboot after every n'th request. That way, it controls memory leaks from getting out of hand. Try from something like 1000 and tune it to your need. –  Grumpy Dec 25 '12 at 8:34
    
@Peter: Thanks! I didn't know about that. Though you're right and it's not really a solution, it may allow me to sleep a little easier while I debug. :) –  Chris Cooper Dec 25 '12 at 19:27
    
MaxRequestsPerChild only works if using embedded mode and WebFaction default configuration for mod_wsgi is these days daemon mode I believe. –  Graham Dumpleton Dec 26 '12 at 5:16
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.