1

we have a mail service with these details:

    1-Centos 6.4
    2:Postfix 2.6.6
    3:roundcube 0.8 
    4:dovecot 2.0.9.7
    5:mysql-server 5.1.71

everything is ok but in peak usage time roundcube sleeped connections increases from 1 or 2 or 3 to 270 in less than 10 minutes and apache opened files (measured by lsof) increase from 4000 to 20000 in that peak time.

this is apache conf: (apache works in prefork mode)

PidFile run/httpd.pid
Timeout 60
KeepAlive On
MaxKeepAliveRequests 100
<IfModule prefork.c>
StartServers       8
MinSpareServers    5
MaxSpareServers   20
ServerLimit      256
MaxClients       256
MaxRequestsPerChild  4000
</IfModule>
TraceEnable off
LimitRequestLine 1024
LimitRequestFields 100
LimitRequestFieldsize 1024
LimitRequestBody 10241024

and here is mysql config:

secure_auth=1
local_infile=0
max_connections        = 600
max_allowed_packet    = 16M
key_buffer        =256M
wait_timeout=240
interactive_timeout=180
connect_timeout=10
innodb_buffer_pool_size=2G

when sleeped connections of roundcube increased to >100 ,almost services (web,mail,mysql) go down....

thanks for any suggestion.

9
  • 1. How did you install your php? 2. How much your RAM/CPU? free -m;cat /proc/cpuinfo; Apr 10, 2014 at 9:09
  • You have decide use prefork or apache worker , so read the following link : codebucket.co.in/apache-prefork-or-worker Apr 10, 2014 at 9:13
  • mysql down or apache ? or all of your machine ....? Apr 10, 2014 at 9:16
  • install php from repository (php 5.2.10) 256G RAM 64 Cores CPU 2.4GHz Apr 10, 2014 at 12:31
  • in peak time all services go down Apr 10, 2014 at 12:32

2 Answers 2

0

The Answer is:

I have edit apache max_client option to lower value 256 --> 50 why!?

for (still) unknown issue all the preforked apache processes take CPU usage about 100% (100% usage of that core running preforked apache process for few moments)

So system go Down , because system has 64 CPU cores when all the 256 processes of apache use 100% cpu usage , system and services go down

issue still exist but services has no problem I think issue related to network attacks (our monitoring tools report many attacks per day), that sometimes make problem such as resource locking or something else

thank you for all suggestion.

0

Now

After about 5 years

The problem has been detected and solved in few days.

It was so complicated for a Jr. System administrator like me ;)

There was a problem in GFS2 cluster file-system that my teammate prepared on iSCSI LUN and this issue led to various issues and problems in Dovecot and roundcube (and then apache)

for your information , when I pay attention to %wa parameter on top command , (it was about 90% ) , I thought (perhaps) there is problem in filesystem level.

Then I had decide to transfer all data to new cluster filesystem (ocfs2) because GFS was deprecated !

First of all, All data moved to new cluster file-system (on ocf2) then re-design whole system based on pacemake haproxy on debian wheezy!

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .