Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have a strange behaviour in my server :-/. Is a OpenVZ VPS (I think is OpenVZ, because /proc/user_beancounters exists and df -h returns /dev/simfs drive. Also ifconfig returns venet0). When I do cat /proc/stat, I can see how each second about 50-100 processes are created and happens about 800k-1200k context switches! All that info is with the server completely idle, no traffic nor programs running.

Top shows 0 load average and 100% idle CPU.

I've closed all non-needed services (httpd, mysqld, sendmail, nagios, named...) and the problem still happens. I do ps -ALf each second too and I don't see any changes, only a new ps process is created each time and the PID is just the same as before + 1, so new processes are not created, so I thought that process growing in cat /proc/stat must be threads (Yes, seems that processes in /proc/stat counts threads creation too as this states: http://webcache.googleusercontent.com/search?q=cache:8NLgzKEzHQQJ:www.linuxhowtos.org/System/procstat.htm&hl=es&tbo=d&gl=es&strip=1).

I've changed to /proc dir and done cat [PID]\status with all PIDs listed with ls (Including kernel ones) and in any process voluntary_ctxt_switches nor nonvoluntary_ctxt_switches are growing at the same speed as cat /proc/stat does (just a few tens/second), Threads keeps the same also.

I've done strace -p PID to all process too so I can see if any process is crating threads or something but the only process that has a bit of movement is ssh and that movement is read/write operations because of the data is sending to my terminal.

After that, I've done vmstat -s and saw that forks is growing at the same speed processes in /proc/stat does. As http://linux.die.net/man/2/fork says, each fork() creates a new PID but my server PID is not growing!

The last thing I can think of is that all process data that proc/stat and vmstat -s show is shared with all the other VPS stored in the same machine, but I don't know if that is correct... If someone can throw some light on this I would be really grateful.

share|improve this question
    
What kind of VPS? Xen, VMWare etc. –  chx Dec 1 '12 at 23:57
    
I think OpenVZ, because /proc/user_beancounters exists and df -h returns /dev/simfs drive. Also ifconfig returns venet0. –  Jorge Fuentes González Dec 2 '12 at 0:49
add comment

2 Answers 2

up vote 1 down vote accepted
+50

In your case I would assume that your kernel is not virtualizing some or all values in /proc/stat, so what you're seeing should be caused by activity in other containers or in the hardware node itself.

You haven't mentioned your specific kernel version so we can't check for sure, but a similar issue was discussed in http://forum.openvz.org/index.php?t=msg&goto=44507& .

PS: check out /proc/vz/vestat instead.

share|improve this answer
    
My kernel is Linux version 2.6.32-6-pve. Reading the forum you linked, finally I've reached the same conclusion, OpenVZ is not isolating /proc/stat. –  Jorge Fuentes González Dec 5 '12 at 23:57
add comment

I have seen at least one similar question here about someone with 2 identical ubuntu servers behind a load balancer...and I will tell you what I told him:

syspro ("system profiler")

or try stuffing some limits in /etc/security/limits.conf Apply the limits to both machines and compare which executables show up. It will at least flag a process or thread and narrow the list of suspects.

man limits.conf: hard & soft for file descriptors, mem locked, stack size,cpu time, nice levels, etc

share|improve this answer
    
I don't think you understood the question. I don't want to limit. I want to know why all that happens. –  Jorge Fuentes González Dec 8 '12 at 10:59
    
I added a clarification. Put the same limits for the same programs and see which programs get reported and HOW MUCH they are over limits. Some disparity should emerge –  ArrowInTree Dec 8 '12 at 13:35
    
Hmmm... Nice idea. Going to test it, thank you. –  Jorge Fuentes González Dec 8 '12 at 14:28
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.