0

Background: I have a very old site running on a very old VPS, a very old Apache and a very old PHP that I can't upgrade because it's a very old CentOS machine. It's an up-to-date WordPress site that serves about 100 pages/minute in a suExec setup. Yes, 100/minute.

Today a site went down for about 15 minutes, starting at 7:01 pm. The first relevant message in the error log file is this:

[Fri Aug 01 19:01:41 2014] [error] [client: a.b.c.d] Premature end of script headers: index.php, referer: ...

There are about two or three hundred more error messages in the log file, ending at 7:02. Google Analytics tells me that the site kept responding to 1 or 2 requests/minute. At 7:15, it started responding normally again, without any intervention from me. The error log file has the normal Apache notices about starting up, like this:

Fri Aug 01 19:16:17 2014] [notice] suEXEC mechanism enabled (wrapper: /usr/local/apache/bin/suexec)

I'm at a loss as to what's causing this. Could it be the server or Apache or PHP running out of memory? Running top gives this for memory, so it seems like it should be something else:

Mem:   1048576k total,   472756k used,   575820k free,        0k buffers
Swap:        0k total,        0k used,        0k free,        0k cached

We need to get through the next two weeks, at which point we will be migrating to a new host. We're ditching the VPS at that point, but we need to squeeze as much reliability out of the old crap as we can until then.

  • Anything in dmesg? – Andrew B Aug 2 '14 at 4:58
  • Just some firewall messages like this: Firewall: TCP_IN Blocked IN=venet0 OUT= PHYSIN=eth0 MAC= SRC=w.x.y.z DST=a.b.c.d LEN=60 TOS=0x00 PREC=0x00 TTL=43 ID=51340 DF PROTO=TCP SPT=51416 DPT=23 WINDOW=5840 RES=0x00 SYN URGP=0 – Michael Aug 2 '14 at 5:16
1

Just add a monitor that sends a dynamic request to the server, decides if it's responded fast enough, and restarts it if not. Considering that your servers are either crashing or going into this 1/minute mode, it should be relatively easy to detect an unhealthy server.

You really should never run "very old" Apache or PHP. They both are subject to security holes. If you don't update them on a daily basis, you are asking for trouble, because hackers can just run last week's security hole against your server and then they have all of your servers and an entry point into the rest of your network.

  • 1
    The monitor is a good idea, thanks. I didn't set this server up, and I haven't been maintaining it for the last seven years. I don't want to run very old software, but right now we don't have a choice. – Michael Aug 2 '14 at 5:18

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.