1

I have an Ubuntu EC2 instance (t1.micro, 600mb of RAM) running mod_wsgi on Apache, to serve a Django app. This is all behind a load balancer.

The problem I keep running into is that I can only seem to get a few days of uptime before my site goes down and I start getting 503 statuses. My record is around 4 weeks with no downtime. Restarting apache does not help, I usually have to completely stop and restart the EC2 instance.

I've tried using Dowser to diagnose memory leaks, but nothing stands out. I've tried tweaking the mpm_prefork settings. I am not using any C extension modules, any external API calls have timeouts set and I only have two cron jobs that run once a day. My access logs don't show anything out of the ordinary that would indicate DDOSing.

I'm completely at a loss as to what's causing the server to go down constantly.

Here is the output from apachectl status during a downtime episode:

enter image description here

And here is the output from htop at the same time:

enter image description here

These are my prefork settings:

<IfModule mpm_prefork_module>
    StartServers              5
    MinSpareServers           5
    MaxSpareServers          10
    MaxRequestWorkers         40
    MaxConnectionsPerChild   0
</IfModule>

Should I just upgrade to a larger EC2 instance? Or upgrade mod_wsgi (since I am on an older version)? I've tried everything from the Django angle so I'm starting to think I just have mod_wsgi configured incorrectly...

4
  • What about the database? Is there a database, and if so, where (same machine?) and which database platform? Sep 14, 2015 at 0:27
  • I'm using an RDS MySQL instance, not on the same machine.
    – kayeight
    Sep 14, 2015 at 4:51
  • Okay, so that shouldn't be the issue. "Here is the output from apachectl status during a downtime episode" it looks like apache had been restarted just a few seconds prior. Was that you? If so, that's not best described as "during" since you've already intervened. What's in the apache logs? You should be seeing ELB health checks, and they should be getting a 200 OK response. The 503 should go away once the machine responds correctly to enough probes. You'll need to find those and verify this behavior. Sep 14, 2015 at 6:22
  • You're right, I had restarted just before that command... whoops. I checked my access logs and for some reason, during the span of the downtime all ELB health checks were getting 200 responses... yet ELB access logs and every other HTTP access around that time show 500 or 503 responses.
    – kayeight
    Sep 15, 2015 at 1:16

0

You must log in to answer this question.

Browse other questions tagged .