My VPS not responding, after reboot everything's fine though. Why?

Question

The server suddenly seems to be very busy and not responding to anything: ssh or web requests.

It's automatically happened several times so apparently something's wrong with my system.

After reboot everything's back to normal and websites are fast and responsive. But approx 24 hours later, it suddenly goes into the same trouble again - very busy and occupied, not responding to anything, took dozens of minutes to just reboot.

I've no idea what may be the trigger of this. Misconfiguration? My PHP code? But all my websites have been fine on the previous host.

My distro is Debian 5.0. Things I've installed:

1. apache, mysql, php
2. rsnapshot (some cron jobs)
3. postfix
4. vsftpd (though automatically stopped)
5. chkrootkit
6. fail2ban

Anyone have any clue? What could be the problem? What logs do I need to see to find out which php script may have caused the problem?

Answer 1

Most likely a memory leak. This can sometimes cause the kind of problems that you have described. If you have swap memory, heavy swapping can severely slow things down to the extent that it is virtually unusable. Otherwise, an OOM error will kill random applications and cause the system to fail as well.

Your best bet would be to check your logs and see if there are any kernel OOM errors. Otherwise, you may want to consider monitoring your memory usage to see if it grows too large over time. Once you have verified that it is insufficient memory, you can then proceed to trace the offending programme causing it.

Update:

The only way to avoid swapping is to reduce the memory foot-print. One way of doing that is by changing your applications and settings. Some things that you may want to consider:

1. Dump Apache for a light-weight web-server like lighttpd/nginx. Reduce the number of php instances it spawns when handling scripts.
2. Configure MySQL using the example /usr/share/doc/mysql-server-5.0/example/my-small.cnf configuration file. Disable networking, innodb, bdb to reduce foot-print.
3. Configure PHP to limit the memory from the default 128Mb to just what you need.
4. Postfix can suck up resources when it spawns multiple processes to handle the memory. Consider going over to qmail or out-sourcing your email server.

Cheers.