Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I am having overloading problems. Too many connections, and some IP has more than 20 connection at once. I do this command.

netstat -anp |grep 'tcp\|udp' | awk '{print $5}' | cut -d: -f1 | sort | uniq -c | sort -n

To get total of connection and this is the output:

      1 106.3.98.81
      1 106.3.98.82
      1 108.171.251.2
      1 110.85.103.207
      1 111.161.30.217
      1 113.53.103.55
      1 119.235.237.20
      1 124.106.19.34
      1 157.55.32.166
      1 157.55.33.49
      1 157.55.34.28
      1 175.141.103.239
      1 180.76.5.59
      1 180.76.5.61
      1 188.235.165.216
      1 205.213.195.70
      1 216.157.222.25
      1 218.93.205.100
      1 222.77.209.105
      1 27.153.148.109
      1 27.159.194.242
      1 27.159.253.71
      1 54.242.122.201
      1 61.172.50.99
      1 65.55.24.239
      1 71.179.78.5
      1 74.125.136.27
      1 74.125.182.30
      1 74.125.182.36
      1 79.112.225.39
      1 93.190.139.208
      2 124.227.191.67
      2 157.55.33.84
      2 157.55.35.34
      2 190.66.3.107
      2 203.87.153.38
      2 220.161.119.3
      2 221.6.15.156
      2 27.153.148.116
      2 27.159.197.0
      2 96.47.224.42
      3 202.14.70.1
      3 218.6.15.42
      3 222.77.218.226
      3 222.77.224.187
      3 37.59.66.100
      3 46.4.181.244
      3 87.98.254.192
      3 91.207.8.62
      4 188.143.233.222
      4 218.108.168.166
      4 221.12.154.18
      4 93.182.157.8
      4 94.142.128.183
      5 180.246.170.187
      5 8.21.6.226
      6 178.137.94.87
      6 218.93.205.112
      7 199.15.234.222
      9
      9 125.253.97.6
     10 178.137.17.196
     11 46.118.192.179
     12 212.79.14.14
     21 72.201.187.135
     27 0.0.0.0

Can anyone give me some directions, my server crashed few times this week because of this.

Thanks.

EDIT:

Alright, my error logs says:

[Thu Oct 18 12:17:39 2012] [error] could not make child process 4842 exit, attempting to continue anyway
[Thu Oct 18 12:17:39 2012] [error] could not make child process 4843 exit, attempting to continue anyway
[Thu Oct 18 12:17:39 2012] [error] could not make child process 4855 exit, attempting to continue anyway
[Thu Oct 18 12:17:39 2012] [error] could not make child process 4856 exit, attempting to continue anyway
[Thu Oct 18 12:17:39 2012] [error] could not make child process 4861 exit, attempting to continue anyway
[Thu Oct 18 12:17:39 2012] [error] could not make child process 4869 exit, attempting to continue anyway
[Thu Oct 18 12:17:39 2012] [error] could not make child process 4872 exit, attempting to continue anyway
[Thu Oct 18 12:17:39 2012] [error] could not make child process 4873 exit, attempting to continue anyway
[Thu Oct 18 12:17:39 2012] [error] could not make child process 4874 exit, attempting to continue anyway
[Thu Oct 18 12:17:39 2012] [error] could not make child process 4875 exit, attempting to continue anyway
[Thu Oct 18 12:17:39 2012] [error] could not make child process 4876 exit, attempting to continue anyway
[Thu Oct 18 12:17:39 2012] [error] could not make child process 4880 exit, attempting to continue anyway
[Thu Oct 18 12:17:39 2012] [error] could not make child process 4882 exit, attempting to continue anyway
[Thu Oct 18 12:17:39 2012] [error] could not make child process 4885 exit, attempting to continue anyway
[Thu Oct 18 12:17:39 2012] [error] could not make child process 4897 exit, attempting to continue anyway
[Thu Oct 18 12:17:39 2012] [error] could not make child process 4900 exit, attempting to continue anyway
[Thu Oct 18 12:17:39 2012] [error] could not make child process 4901 exit, attempting to continue anyway
[Thu Oct 18 12:17:39 2012] [error] could not make child process 4906 exit, attempting to continue anyway
[Thu Oct 18 12:17:39 2012] [error] could not make child process 4907 exit, attempting to continue anyway
[Thu Oct 18 12:17:39 2012] [error] could not make child process 4925 exit, attempting to continue anyway
[Thu Oct 18 12:17:39 2012] [error] could not make child process 4926 exit, attempting to continue anyway
[Thu Oct 18 12:17:39 2012] [error] could not make child process 4927 exit, attempting to continue anyway
[Thu Oct 18 12:17:39 2012] [error] could not make child process 4931 exit, attempting to continue anyway
[Thu Oct 18 12:17:40 2012] [notice] caught SIGTERM, shutting down
PHP Warning:  PHP Startup: Unable to load dynamic library '/usr/lib/php5/20060613+lfs/curl.iso' - /usr/lib/php5/20060613+lfs/curl.iso: cannot open shared object file: No such file or directory in Unknown on line 0
[Thu Oct 18 12:17:45 2012] [notice] Apache/2.2.9 (Debian) PHP/5.2.6-1+lenny10 with Suhosin-Patch configured -- resuming normal operations

And I have over thousands of line saying:(each has different process id)

[Thu Oct 18 12:17:38 2012] [error] child process 4906 still did not exit, sending a SIGKILL

And I also have line saying:

[Wed Oct 17 09:44:58 2012] [error] server reached MaxClients setting, consider raising the MaxClients setting


<IfModule prefork.c>
StartServers 8
MinSpareServers 5
MaxSpareServers 50
MaxClients 300
MaxRequestsPerChild 5000
</IfModule>
share|improve this question
2  
How do you know your "server crashed" due to this? A few hundred connections should not cause any issues even on ancient hardware. What does the error log say ? Did you investigate server-status ? –  adaptr Oct 18 '12 at 10:31
1  
PHP is a good hint. My guess is that your PHP application is doing something that takes a long time or hanging at one part and you are getting requests faster than you can process them. My approach is usually to run strace -f -p <Apache PID> -s 500 -T -tt on one of the Apache children and analyse the output to see what's taking the time. There are also PHP debuggers you can use but I wouldn't do that in production. Since curl and include are also common causes, running a tcpdump may help identify the problem. Also, your PHP error log. –  Ladadadada Oct 18 '12 at 10:58
    
@adaptr Sometime I have 700 from same IP and it usually stops apache from working. It seems like a similar situation now. –  Alice Oct 18 '12 at 11:00
    
@Ladadadada I am just using vBulletin though. My php error log was off, just turned it on. –  Alice Oct 18 '12 at 11:42
    
It's possible you have the cause and effect backwards. You may have all these connections because you are overloaded. (If the line is long at Burger King and service is slow, it's not clear which is the cause and which is the effect. The line might be long because service is slow, say because they're understaffed. Or service might be slow because the line is long, say because a bus full of Boy Scouts just got there.) You need to do more work towards understanding what's going on before you try to formulate a fix. (Punch vBulletin "attempting to continue anyway" into a search engine.) –  David Schwartz Oct 18 '12 at 13:54
add comment

1 Answer

That command ignores the state of the connection. The following command only shows the connections in the ESTABLISHED state.

netstat -anp |grep 'tcp\|udp' | awk '/ESTABLISHED/ {print $5}' | cut -d: -f1 | sort | uniq -c | sort -n

This one groups and aggregates them:

netstat -tn | awk '/tcp/ {print $5, $6}' | sed 's/:[^ ]*//' | sort | uniq -c | sort -n

A couple more questions:

  1. What do you mean by "my server crashed" ? OS reboot? Apache not responding? Exactly what happened.
  2. How do you know that the number of connections was what caused the "crash"? If you have an error message that states "Too many connections; aborting." that would be good. If not, let's investigate before jumping to conclusions.
share|improve this answer
    
Hey, thanks for fast reply. By crashed I mean not be able to access my site and I had to contact my provider for a reboot. Putty wouldn't even let me access. I posted more info above, I think it explains why it crashed, it says max-client reached. –  Alice Oct 18 '12 at 11:08
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.