Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

i have several cron jobs running on an ubuntu machine. each one does some pretty heavy load stuff. The cron jobs are parsing files and the bigger the file the longer it takes them to parse it. The strange thing is that if i make the files too big ( like 30mb) the script kind of hangs. It starts processing them really enthusiastically but after some time (something like 5-10 minutes) the cpu usage of the process drops a lot and it gets into some "zombie" state. If prior to this the process in htop was using 70-80% of the CPU then after this drop occurs it slows down to something like 5-10%. the load average drops down as well. The status of the processes sometimes changes to D in htop, which AFAIR stands for zombie. Today i noticed the same behavior of processes of mysql when executing heavy queries (a query took something like 4 hours to execute). the cron jobs are mostly php and during their processing most of the CPU eats the php process and not mysql. so i think the issue is not with a specific language/program but with the way the processes are "managed".

The only other place i've seen similar behavior was on my Amazon EC2 micro instance when after some aggressive use of CPU the CPU quota was taking effect and everything was slowing down dramatically.

This is a dedicated machine running ubuntu. what may be the cause?

Edit: adding some details

the memory is okay and it's not an issue of swapping IMHO.

iostat says this about IO activity: http://img13.imageshack.us/i/captureehm.png/ it looks okay to me cause some IO was expected to take place and it looks like the processor is not "overwhelmed" by IO waiting. correct me if i'm wrong :)

share|improve this question
    
I have had situations where a cron job took so long that the next one started running before the last had finished (to the detriment of both!). Could that be the case here? –  David Jan 13 '11 at 23:11
    
no .. this happens when just one task is running –  Yervand Aghababyan Jan 14 '11 at 16:48
add comment

3 Answers

sar(1) will provide the data you need to analyze here. You can look at the sar -A to see all collected sar stats, then drill down. For example sar -b will give you the I/O stats to see if you are getting bogged down on disk activity.

A nice feature of sar is it logs in the background so you can use it to review historical data, not just current stats.

share|improve this answer
add comment

Well, you can check to see if it's being "niced", but it sounds more like the process is bogging down due to memory or IO issues. Can you post the memory usage?

share|improve this answer
    
it's not being niced. all processes besides 3 udev ones are niced to 0. the memory is also okay. most of it is empty. maybe the cause is slow IO, i thought about it. but i'm nor sure how can i check this? –  Yervand Aghababyan Jan 12 '11 at 17:36
    
@yervand: Try iostat –  Satanicpuppy Jan 12 '11 at 17:41
    
iostat says this: img13.imageshack.us/i/captureehm.png it looks okay to me. am i wrong? –  Yervand Aghababyan Jan 12 '11 at 17:57
    
@yervand: Hmmm, yea, looks fine. –  Satanicpuppy Jan 12 '11 at 19:47
add comment

The "D" state means the command is waiting on IO. It's not quite the same as a zombie process.

Check how much ram your process is using - if it's using too much and is swapping out, this could well manifest as the symptoms you're seeing.

share|improve this answer
    
To clarify: a zombie process is one that has exited but still exists in the process table so that it's parent can retrieve the exit status. –  Phil Hollenback Jan 12 '11 at 17:45
    
thank you for the explanation :) –  Yervand Aghababyan Jan 12 '11 at 17:59
    
To add to what Phil said: a zombie takes up no memory and uses no resources whatsoever - except the process exit value stored in the process table. Zombies are a pain, but they shouldn't affect system operation adversely. –  David Jan 13 '11 at 23:08
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.