Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

We have a server with a memory issue: the server keeps allocating itself memory and doesn't release it. We're running Apache. I set MaxReqsPerClient to a really low value just so the threads don't hold a lot of memory, but has anyone seen calls like this? Am I wrong in thinking that it's probably Drupal pulling too much data back from the cache in DB?

read(52, "h_index\";a:2:{s:6:\"weight\";i:1;s"..., 6171) = 1368
read(52, "\";a:2:{s:6:\"author\";a:3:{s:5:\"la"..., 4803) = 1368
read(52, ":\"description\";s:19:\"Term name t"..., 3435) = 1368
read(52, "abel\";s:4:\"Name\";s:11:\"descripti"..., 2067) = 1368
read(52, "ions\";a:2:{s:4:\"form\";a:3:{s:4:\""..., 16384) = 708
brk(0x2ab554396000)                     = 0x2ab5542f5000
mmap(NULL, 1048576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2ab55f653000
brk(0x2ab554356000)                     = 0x2ab5542f5000
mmap(NULL, 1048576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2ab55f753000
brk(0x2ab554356000)                     = 0x2ab5542f5000
mmap(NULL, 1048576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2ab55f853000
brk(0x2ab554356000)                     = 0x2ab5542f5000
mmap(NULL, 1048576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2ab55f953000
brk(0x2ab554356000)                     = 0x2ab5542f5000
mmap(NULL, 1048576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2ab55fa53000
brk(0x2ab554356000)                     = 0x2ab5542f5000
mmap(NULL, 1048576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2ab55fb53000
brk(0x2ab554356000)                     = 0x2ab5542f5000
mmap(NULL, 1048576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x2ab55fc53000
poll([{fd=52, events=POLLIN|POLLPRI}], 1, 0) = 0 (Timeout)
write(52, "d\0\0\0\3SELECT cid, data, created, "..., 104) = 104
read(52, "\1\0\0\1\5E\0\0\2\3def\23drupal_database_nam"..., 16384) = 1368
read(52, ";s:11:\"granularity\";a:5:{s:4:\"ye"..., 34783) = 1368
read(52, ":4:\"date\";}s:9:\"datestamp\";a:9:{"..., 33415) = 1368
read(52, "\";i:0;s:15:\"display_default\";i:0"..., 32047) = 1368
read(52, "e as an integer value.\";s:8:\"set"..., 30679) = 1368
read(52, "label' pairs, i.e. 'Fraction': 0"..., 29311) = 1368

top (the procs just keep growing in memory..):

12845 apache    15   0  581m 246m  37m S  0.0  4.1   0:17.39 httpd                                   
12846 apache    15   0  571m 235m  37m S  0.0  4.0   0:12.13 httpd                                   
12833 apache    15   0  420m 117m  37m S  0.0  2.0   0:06.04 httpd                                   
12851 apache    15   0  412m 113m  37m S  0.0  1.9   0:05.32 httpd                                   
13871 apache    15   0  409m 109m  37m S  0.0  1.8   0:04.90 httpd                                   
12844 apache    15   0  407m 108m  37m S  0.0  1.8   0:04.50 httpd                                   
13870 apache    15   0  407m 108m  37m S  0.3  1.8   0:03.50 httpd                                   
14903 apache    15   0  402m 103m  37m S  0.3  1.7   0:01.29 httpd                                   
14850 apache    15   0  397m 100m  37m S  0.0  1.7   0:02.08 httpd                                   
14907 apache    15   0  390m  93m  36m S  0.0  1.6   0:01.32 httpd                                   
13872 apache    15   0  386m  91m  37m S  0.0  1.5   0:03.13 httpd                                   
12843 apache    15   0  373m  81m  37m S  0.0  1.4   0:02.51 httpd                                   
14901 apache    15   0  370m  75m  33m S  0.0  1.3   0:00.78 httpd                                   
14904 apache    15   0  335m  29m  15m S  0.0  0.5   0:00.26 httpd  
share|improve this question

2 Answers 2

Over what time period is this occurring? How big does memory usage get?

If it's a short-enough time, or if you don't running Apache in the foreground for a bit, you could use valgrind to find memory leaks... (It's available via YUM)

Something like:

valgrind --leak-check=full /usr/sbin/httpd 

The output could potentially show something like:

==945== LEAK SUMMARY:
==945== definitely lost: 15,595 bytes in 120 blocks
==945== indirectly lost: 1,917,248 bytes in 1,242 blocks
==945== possibly lost: 0 bytes in 0 blocks
==945== still reachable: 11,743 bytes in 89 blocks
==945== suppressed: 0 bytes in 0 blocks
==945== Reachable blocks (those to which a pointer was found) are not shown.
==945== To see them, rerun with: --leak-check=full --show-reachable=yes

You can view the memory allocation of a process with the pmap utility.

For Apache, I'd take a look at the PID. Obtain the PID value with: cat /var/run/httpd.pid or just run:

pmap -x $(cat /var/run/httpd.pid)

But in my limited Apache experience, leaks always seem to be tied to modules and scripting languages. Could be caching, too...

share|improve this answer
    
Memory usage increases until the box starts swapping and everything grinds to a halt. Then our phones light up with complaining users :P I'll have to look into valgrind; I hadn't heard of it. I also did a bit of extra testing, and the cache queries are only using about 522K, so unless a bunch of cache queries are going off at once (which is entirely possible), then idk if that'd be the problem. I need to log my mysql queries for a bit to see if those cache views are being called frequently. –  Zipp Dec 11 '12 at 21:51
up vote 0 down vote accepted

I ended up just lowering MaxRequestsPerChild in httpd.conf so my threads expire before they get too big. They were previously getting up to 800M and there was some serious swapping, and now they get to at most 250M before they die.

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.