I'm getting seemingly random server hangs (specifically OOM-ing) and running out of skill trying to track this down.
I'm using a Debian 5 VS with Apache/mySQL/PHP. I've got Postfix running using mySQL as well.
I was ssh-ed in when it happened last used top to see: 1. load average shot up over 25 and higher 2. CPU as 49.8%wait, 48.6%idle, so some kind of IO blocking? 3. 13 apache2 processes, totalling 41.4% of memory 4. mySQL showing only 2.6% of memory
Memory showed: Mem: 524512k total, 518144k used, 6368k free, 800k buffers Swap: 262136k total, 261024k used, 1112k free, 22824k cached
I've got Munin installed and it doesn't show (to my inexperienced eyes) anything really pathological happening at the time this happened - even postfix isn't doing much in terms of queue size
df tells me I'm only using 58% of my disk so I'm not close to topping out there.
php.ini is set to 128M max memory, 30second max execution time
I've been dredging through apache and mySQL logs, but can't see anything.
Can anyone suggest a next step in terms of what extra monitoring I could put on the server, or further logging?