Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have an Amazon EC2 micro instance running an old 2.6.16 kernel. It runs postfix, apache, and mysql. During normal loads, it's load average is around 0.05, and it runs this way 95% of the time or so. However, a few times a day (or so), the CPU usage will spike to 100% and the system becomes nearly unusable. This usually lasts for roughly 5 minutes, then the load returns to normal.

If I manage to take a look at htop while this happens (not easy -- the load is that severe), I see that no running task accounts for any significant cpu usage, leading me to believe this must all be taking place in kernel-land.

How can I diagnose the cause of this load and, more importantly, fix it?

share|improve this question

1 Answer 1

up vote 1 down vote accepted

What is the percentage of "iowait" and "steal" CPU time during these periods?

Iowait denotes the amount of time the CPU is spending waiting for IO requests to complete, and steal percentage denotes CPU time that your kernel requested, but was denied by the hypervisor.

EC2 t1.micro instances are very CPU and IO-constrained. They can burst for very short amounts of time, after which they're subject to severe CPU throttling. Next time this happens, pay attention to %wa and %st in the output of top. My bet is that one or both of these have high percentages of CPU time.

To mitigate, you'll need to find the source of the IO and/or CPU load or alternatively, resize your instance to an m1.small.

share|improve this answer
    
Conveniently, this just happened again. %st is pegged at 98% or so. –  stephelton Apr 19 '13 at 18:48
    
@stephelton - yep, that's Xen's CPU throttling kicking in. That's one reason why t1.micro instance are no good for anything but the most light workloads, or applications where performance is a very low priority. –  EEAA Apr 19 '13 at 18:49
    
Yep. Related question: serverfault.com/questions/230495/what-does-st-mean-in-top –  stephelton Apr 19 '13 at 18:50
1  
I suggest you edit your answer to include brief explanation of what %st means in the context of EC2/virtualization. Anyway, thanks! –  stephelton Apr 19 '13 at 18:51

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.