2

I have server with CentOS 6.2, apache 2.2, php 5.3, mysql 5.1. Server is installed in KVM container.

From time to time apache stalls. It just stops accepting connections. Looks like it reaches ServerLimit and keep accepted connections opened.

The strange thing in my opinion is that after service httpd restart server goes down again very fast. It accepts connections only for 1-2 seconds may be. And stalls again. But after reboot it can work several hours without problems. Or can stall again in several minutes. But it is not seconds like after service httpd restart.

Another noticeable thing is that after server stalled, I'm not able to connect to it via ssh. But if I already was connected, I can type commands.. but, if I type i.e. service httpd restart system performs my command very slow (but CPU is free and memory is rather free). So, it looks like server cant allocate some resource and waits for it. But I'm not sure what exactly it waits for..

there are variables from sysctl -a, which were changing during server working

servername/vars/

Server settings

ServerLimit 140
MaxClients 128
MinSpareServers 5
MaxSpareServer 10
MaxRequestsPerChild 4000
KeepAlive On
KeepAliveTimeout 2
MaxKeepAliveRequests 10

Setting keepAlive to off didnt help


update

I think i know what is wrong. Looks like host's SELinux by some reasons randomly blocks kvm-guest's writing to disk. May be it is because i didnt use the libvirt pool, but just lvm-volumes to add disks devices to guest. now I need only wait while it will happen again and check logs to be sure.

update 2

NO, it is not SELinux. Question is still opened. Any ideas?

6
  • What is that http://174.36.251.141/vars/ link? Also, I'd suggest looking into possible networking issues that may cause this. Perhaps capturing some packets just to make sure it isn't network related. Mar 10, 2012 at 19:54
  • What is that 174.36.251.141/vars link?
    – Andrey
    Mar 10, 2012 at 20:37
  • Hm.. yes that is what I asked :-) Mar 10, 2012 at 20:41
  • :) What is that 174.36.251.141/vars link? it is the dynamics of system variables, if do sysctl -a + netstat -n | grep -c WAIT + netstat -n | grep -c ESTAB, but only for changing values (so, only those values which were changed during monitoring, will be shown). Every graph has the name. It is the name of sysctl variable (or TIME_WAIT and ESTABLISHED for netstat) Perhaps capturing some packets just to make sure it isn't network related. How to do it? I can say if i already connected via ssh, i can type after apache stalls. And httpd restart doesnt help, only reboot.
    – Andrey
    Mar 10, 2012 at 20:45
  • also, i have access to host server and can check network from there. Network looks ok. I can connect to host at least. And some time ago it was another KVM container with the same problem but another ip address... and the same behaviour. So, i think it is not network. At least not external network.
    – Andrey
    Mar 10, 2012 at 20:48

2 Answers 2

0

Try lowering: MinSpareServers 5 and MaxSpareServer 10. Also tell us how much RAM do you have ?

0

This was a problem with iptables. I'm not sure what was it exactly and why... But as fast as I have moved rule -A INPUT -i lo -j ACCEPT to the top of all other rules, it started work correctly and no this problem again ever (before this the first rule was about eth0).

2
  • If you suspect it is iptables' problem, could you post the iptables rules? Dec 25, 2013 at 16:41
  • It was about 2 years ago from now. I dont remember the details. But I can say that the first rule was like this: -A INPUT -i eth0 -m state --state ESTABLISHED,RELATED -j ACCEPT, and now the first rule is -A INPUT -i lo -j ACCEPT. Also one more detail, apache was behind nginx. So, nginx has listened main ip address and apache listened only 127.0.0.1. And nginx was sending php requests to apache and served static files itself.
    – Andrey
    Dec 26, 2013 at 6:19

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .