1

I have a memory leak on one of my apps being served with nginx + passenger and a min/max_instances 2 setting. Memory keeps growing up as time passes by, and this can be a problem on a machine with limited memory (eg Amazon Web Services) that can end up crashing badly.

Is there a way / trick to restart some passenger processes of the same app, either periodically or when memory reaches a certain threshold ?

Suppose I have an application group A that can start up to 2 processes A1 and A2. I'd like to restart A1 and A2 one at a time to ensure service continuity (this is not a zero-downtime deployment since it relies on the same unmodified app, but some way to purge the memory leak and restart a fresh process until I figure out the problem).

Eg suppose I have the following state:

Requests in queue: 0

  * PID: 11124   Sessions: 0       Processed: 14638   Uptime: 1d 12h 31m 29s
    CPU: 0%      Memory  : 341M    Last used: 24s
  * PID: 11131   Sessions: 0       Processed: 9323    Uptime: 1d 12h 31m 29s
    CPU: 0%      Memory  : 389M    Last used: 24s

Process n°2 starts taking up a lot of memory (~400) so I'd like to restart it (and only this process) which should bring it back to 100MB ish, and I'd do the same with Process n°1 later

The memory leak takes some time to build and it would be best if I could restart only when detecting too high memory usage, but if I can schedule a process restart every night or so this could also work totally fine (eg restart process n°1 at 3am, then process n°2 at 4am)

Is there any signal I can send to this passenger group that would let me restart processes one by one ?

1

This is may be a little bit to late for the OP, but we have the following cron-script running on our AWS hosts every minute:

#!/bin/bash
let "TARGET = 400 * 1024" # = 400 MB
ps aux --sort -rss | grep "Passenger RackApp" | while read -r LINE; do
        read PID MEMORY <<< $(echo $LINE | awk '{ print $2; print $6 }')
        if [ $MEMORY -gt $TARGET ]
        then
                `kill -15 $PID`
        fi
done
exit 0

Depending on the configuration, passenger will immediately re-spawn the stopped process.

0

If this is your production application, I'd suggest you go with passenger enterprise. There is a nice feature you can add to the nginx config:

passenger_memory_limit 1024;

Passenger will reload the process when the memory hits 1024 or whatever number you want to specify. You can also take advantage of multi-threading since passenger standalone is single threaded.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.