My server is experiencing very high load average spikes (>10, sometimes even >20) every few minutes.
top shows that CPU isn't in usage but is waiting for I/O operations:
top - 17:42:28 up 8 days, 8:10, 1 user, load average: 9.01, 10.16, 6.54
Cpu0 : 0.0%us, 0.0%sy, 0.0%ni, 0.0%id,100.0%wa, 0.0%hi, 0.0%si, 0.0%st
Cpu1 : 0.0%us, 0.0%sy, 0.0%ni, 0.0%id,100.0%wa, 0.0%hi, 0.0%si, 0.0%st
dmesg shows this output over and over again (I don't understand what it means):
ata2.00: exception Emask 0x0 SAct 0x0 SErr 0x0 action 0x2
ata2.00: BMDMA stat 0x26
ata2.00: cmd ca/00:08:74:c4:24/00:00:00:00:00/ef tag 0 dma 4096 out
res 51/84:01:7b:c4:24/84:00:10:00:00/ef Emask 0x30 (host bus error)
ata2.00: status: { DRDY ERR }
ata2.00: error: { ICRC ABRT }
ata2: soft resetting link
ata2.00: configured for UDMA/33
ata2: EH complete
sd 3:0:0:0: [sdb] 625142448 512-byte hardware sectors (320073 MB)
sd 3:0:0:0: [sdb] Write Protect is off
sd 3:0:0:0: [sdb] Mode Sense: 00 3a 00 00
sd 3:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Any ideas?
There is FreeRADIUS running on server and I am suspecting that either RADIUS either network adapter might be causing the problem. During some spikes tcpdump showed increased number of RADIUS packages being sent/received (but I'm talking about tens of packages per minute, not thousands of them).
When I stop RADIUS situation gets better, but there are still periodic load avg spikes (more tolerable though).
Does anybody have an idea what might be causing this behavior, and how I can determine for sure is it RADIUS, network adapter, or something else?
Thanks