Not so far ago LA on my server raised to 400 and I couldn't even login to server using ssh. Does exists any software, that can prevent this situations by automatically killing processes that making huge load on server?
PS. Debian 6.0.5
You can use a watchdog like Monit to watch over the processes you care about, and restart them if they consume excess resources.
Something like this would be used to monitor Apache:
So, if the cpu% for the Apache process or any of its children are over 40%, send an alert. If it's above 80%, do a restart of Apache.
Monit will also start up Apache if it's not running for some reason, which is a reasonable way to keep critical services up (if you don't have something like Upstart available).
This assumes that you have a set of processes that you can target for this sort of monitoring. Presumably, you suspect a particular application may be a problem.
Load is an aggregate value of tasks with processing work left to run.
It is a measure of the remainder of processing left to do after each process has been given a fair time slice of CPU. Effectively it puts a number on how well the system is dealing with competition of resources between processes.
IOWait can also increase load if processes are causing large IO usage which would point to a memory problem (using far too much of it and aggressively swapping) or an underlying I/O problem. If its just one process that uses up loads of I/O it would be punished usually anyway without too much affecting other processes.
I dont think you can just simply kill one process to fix a problem like this as the problem stems from a few processes demands on the CPU. You could kill a process group or a thread pool for one particular process though.
When your LA raises and you can't login via ssh, try Grey Goo a tiny available and reliable remote command execution server and client designed purely for emergency situations: