Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

The server suddenly seems to be very busy and not responding to anything: ssh or web requests.

It's automatically happened several times so apparently something's wrong with my system.

After reboot everything's back to normal and websites are fast and responsive. But approx 24 hours later, it suddenly goes into the same trouble again - very busy and occupied, not responding to anything, took dozens of minutes to just reboot.

I've no idea what may be the trigger of this. Misconfiguration? My PHP code? But all my websites have been fine on the previous host.

My distro is Debian 5.0. Things I've installed:

  1. apache, mysql, php
  2. rsnapshot (some cron jobs)
  3. postfix
  4. vsftpd (though automatically stopped)
  5. chkrootkit
  6. fail2ban

Anyone have any clue? What could be the problem? What logs do I need to see to find out which php script may have caused the problem?

share|improve this question
    
Is it just very very slow or totally inresponsive? Can you ping the box when this happens? –  e-t172 Sep 15 '09 at 12:57
    
so slow that it's totally irresponsive. maybe plain irresponsive, I don't know... –  kavoir.com Sep 15 '09 at 13:42
add comment

1 Answer

up vote 2 down vote accepted

Most likely a memory leak. This can sometimes cause the kind of problems that you have described. If you have swap memory, heavy swapping can severely slow things down to the extent that it is virtually unusable. Otherwise, an OOM error will kill random applications and cause the system to fail as well.

Your best bet would be to check your logs and see if there are any kernel OOM errors. Otherwise, you may want to consider monitoring your memory usage to see if it grows too large over time. Once you have verified that it is insufficient memory, you can then proceed to trace the offending programme causing it.

Update:

The only way to avoid swapping is to reduce the memory foot-print. One way of doing that is by changing your applications and settings. Some things that you may want to consider:

  1. Dump Apache for a light-weight web-server like lighttpd/nginx. Reduce the number of php instances it spawns when handling scripts.
  2. Configure MySQL using the example /usr/share/doc/mysql-server-5.0/example/my-small.cnf configuration file. Disable networking, innodb, bdb to reduce foot-print.
  3. Configure PHP to limit the memory from the default 128Mb to just what you need.
  4. Postfix can suck up resources when it spawns multiple processes to handle the memory. Consider going over to qmail or out-sourcing your email server.

Cheers.

share|improve this answer
    
Hi, thanks for the heads up. My servers are with Linode and seems they have some good answers concerning this: linode.com/forums/viewtopic.php?t=4630 linode.com/wiki/index.php/Swappiness –  kavoir.com Sep 15 '09 at 15:36
    
Do you think following these answers would result in a good server that's much less OOM prone? –  kavoir.com Sep 16 '09 at 2:45
    
There are lots of things you can do to help reduce memory usage. I've added some considerations in the answer. YMMV. –  sybreon Sep 16 '09 at 3:16
    
I second the suggestion that it sounds like you are OOMing. You can confirm this is the case or not by checking the console the next time your Linode is unresponsive and look for OOM killer messages. If that isn't the case definitely open a support ticket so we (I work at Linode ;) can try to help you figure out what is going on. –  mikegrb Sep 16 '09 at 3:23
    
Oops, forgot to include a link, library.linode.com/troubleshooting/memory-networking.html has some additional information of memory issues. –  mikegrb Sep 16 '09 at 3:24
show 3 more comments

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.