Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I have a c1.medium tier server running Ubuntu 12.04 on AWS EC2. We use it as a development server, hosting between 5-8 Drupal 7 sites on it. Recently the server has been performing very slowly and I'm struggling to find why. I've searched logs, but can't detect anything obvious. I've notice whenever the server starts to crawl / crash - looking at the cloud watch graphs on the EC2 console, there is always an unusual spike. Any recommendations on how to investigate and identify these problems and where they're coming from.. I've attached the output of top and a snippet of my apache2.conf file.

Tasks: 117 total,   1 running, 113 sleeping,   3 stopped,   0 zombie
Cpu(s):  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:   1733244k total,  1647596k used,    85648k free,     4612k buffers
Swap:   917500k total,   644984k used,   272516k free,    69512k cached

  PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND                                                                                                                                                
16417 mysql     20   0 1317m 199m 1784 S    0 11.8   3:19.84 mysqld                                                                                                                                                 
18087 www-data  20   0  483m 139m 3132 S    0  8.3   0:11.15 apache2                                                                                                                                                
18019 www-data  20   0  483m 136m 3208 S    0  8.1   0:13.52 apache2                                                                                                                                                
18088 www-data  20   0  477m 131m 2952 S    0  7.8   0:10.31 apache2                                                                                                                                                
17958 www-data  20   0  533m 130m 2948 S    0  7.7   0:16.77 apache2                                                                                                                                                
17959 www-data  20   0  450m 103m 3192 S    0  6.1   2:05.47 apache2                                                                                                                                                
17987 www-data  20   0  443m  97m 3188 S    0  5.8   0:10.76 apache2                                                                                                                                                
17961 www-data  20   0  438m  95m 3180 S    0  5.7   0:13.21 apache2                                                                                                                                                
18089 www-data  20   0  443m  83m 3172 S    0  4.9   0:06.69 apache2                                                                                                                                                
18068 www-data  20   0  412m  68m 3148 S    0  4.0   0:07.99 apache2                                                                                                                                                
18459 www-data  20   0  411m  68m 3204 S    0  4.0   0:01.27 apache2                                                                                                                                                
17960 www-data  20   0  410m  67m 3812 S    0  4.0   0:06.79 apache2                                                                                                                                                
18283 ubuntu    20   0 26720 6908    0 S    0  0.4   0:00.59 bash                                                                                                                                                   
18460 www-data  20   0  350m 2788  804 S    0  0.2   0:00.00 apache2                                                                                                                                                
13391 root      20   0  350m 1700   16 S    0  0.1   0:03.25 apache2                                                                                                                                                
17830 ubuntu    20   0 26768 1468  620 S    0  0.1   0:01.48 bash                                                                                                                                                   
18463 ubuntu    20   0 17332 1288  944 R    0  0.1   0:00.04 top                                                                                                                                                    
 8024 root      20   0 2632m  936    0 S    0  0.1  14:24.58 java                                                                                                                                                   
18134 root      20   0 73352  772    0 S    0  0.0   0:00.06 sshd                                                                                                                                                   
18282 ubuntu    20   0 73352  756    0 S    0  0.0   0:00.03 sshd                                                                                                                                                   
    1 root      20   0 24420  516    0 S    0  0.0   0:13.99 init                                                                                                                                                   
  909 postfix   20   0 27372  348  208 S    0  0.0   0:43.46 qmgr                                                                                                                                                   
  797 root      20   0 15972  320  232 S    0  0.0  25:04.39 irqbalance                                                                                                                                             
  900 root      20   0 25096  272  172 S    0  0.0   1:49.94 master                                                                                                                                                 
  664 syslog    20   0  248m  244    0 S    0  0.0  18:20.90 rsyslogd                                                                                                                                               
16201 ubuntu    20   0 73468  220    0 S    0  0.0   0:00.07 sshd                                                                                                                                                   
16904 ubuntu    20   0 73468  216    0 S    0  0.0   0:00.03 sshd   
Timeout 20
KeepAlive On
MaxKeepAliveRequests 500
KeepAliveTimeout 2
IfModule mpm_prefork_module>
    StartServers          5
    MinSpareServers       5
    MaxSpareServers      12
    MaxClients           40
    MaxRequestsPerChild  10000
/IfModule>

Completely new to all of this.. any advice much appreciated. Cheers

share|improve this question
    
Can you clarify what you mean by "crawl / crash" ? Is it HTTP requests that take a long time? MySQL queries? SSH traffic? Bash commands? That top output looks like a completely idle server. –  Ladadadada Feb 28 '13 at 16:16
    
A bit of a guess with the data provided, but: you have about 160M of free RAM, each new apache process is taking about 70M of RAM. If you launch a few extra apache processes (up to MaxClients), you could conceivably start swapping - and slow to a crawl. sar (sysstat) might get you a quick diagnosis (especially if you already have it setup), but atop will likely provide you with more detailed information. (Also: unusual spike in which graph(s) (IO, RAM, CPU, etc.)? –  cyberx86 Feb 28 '13 at 19:13
add comment

1 Answer

Install atop and run it as a daemon logging to /var/log/atop.log. Then, next time you have a nasty load spike, you can run atop -r /var/log/atop.log and look at the state of the machine at the time it happened (use the t key to fast-forward through the log).

atop is superior to top in the usefulness and completeness of data it shows you.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.