2

I have a Drupal web server vps under centos 6.3 that has been running well since February.

In the last few days, the site has started going down: tcp connects, then it waits indefinitely.

If I restart httpd, the site becomes responsive for a few minutes before grinding to a halt again.

There is plenty of disk space, the cpu is idle, and about half the system memory is in buffers and cache. Mysqld remains responsive.

If I run pstack on the httpd worker processes, they seem to be stuck in the same place:

#0  0x00007f568d9c3a28 in poll () from /lib64/libc.so.6
#1  0x00007f568deb6717 in apr_wait_for_io_or_timeout () from /usr/lib64/libapr-1.so.0
#2  0x00007f568deb1a28 in apr_socket_sendv () from /usr/lib64/libapr-1.so.0
#3  0x00007f568f3d3083 in ?? ()
#4  0x00007f568f3d4035 in ap_core_output_filter ()
#5  0x00007f568f3e6194 in ap_http_chunk_filter ()
#6  0x00007f568f3e46db in ap_http_header_filter ()
#7  0x00007f568f3c8fd0 in ap_content_length_filter ()
#8  0x00007f568f3e6443 in ap_byterange_filter ()
#9  0x00007f568f3df9fc in ap_filter_flush ()
#10 0x00007f568f3c6f8e in ap_rwrite ()
#11 0x00007f5689814f76 in ?? () from /etc/httpd/modules/libphp5.so
#12 0x00007f568974e36b in php_ub_body_write_no_header () from /etc/httpd/modules/libphp5.so
#13 0x00007f568974e998 in php_end_ob_buffer () from /etc/httpd/modules/libphp5.so
#14 0x00007f568974f32f in ?? () from /etc/httpd/modules/libphp5.so
#15 0x00007f568974e998 in php_end_ob_buffer () from /etc/httpd/modules/libphp5.so
#16 0x00007f568974f051 in ?? () from /etc/httpd/modules/libphp5.so
#17 0x00007f56897db0c8 in ?? () from /etc/httpd/modules/libphp5.so
#18 0x00007f56897b2400 in execute () from /etc/httpd/modules/libphp5.so
#19 0x00007f568978cb3d in zend_execute_scripts () from /etc/httpd/modules/libphp5.so
#20 0x00007f568973ada8 in php_execute_script () from /etc/httpd/modules/libphp5.so
#21 0x00007f5689815a85 in ?? () from /etc/httpd/modules/libphp5.so
#22 0x00007f568f3d4b00 in ap_run_handler ()
#23 0x00007f568f3d83be in ap_invoke_handler ()
#24 0x00007f568f3e386c in ap_internal_redirect ()
#25 0x00007f568b82a7a5 in ?? () from /etc/httpd/modules/mod_rewrite.so
#26 0x00007f568f3d4b00 in ap_run_handler ()
#27 0x00007f568f3d83be in ap_invoke_handler ()
#28 0x00007f568f3e3a30 in ap_process_request ()
#29 0x00007f568f3e08f8 in ?? ()
#30 0x00007f568f3dc608 in ap_run_process_connection ()
#31 0x00007f568f3e8807 in ?? ()
#32 0x00007f568f3e8b1a in ?? ()
#33 0x00007f568f3e979c in ap_mpm_run ()
#34 0x00007f568f3c0900 in main ()

Any suggestions where to look next in debugging this?

0

Try running strace -p PID_OF_APACHE -f to debug the output. Also, maybe you need to update your Apache version and/or one or more of its dependencies - or even Drupal if that isn't updated too.

You might want to check with the changelogs of the services above to see if there any known issues.

0

The stack trace shows php flushing the output buffer, and waiting for the browser to respond.

You should make a network trace, see if traffic from certain clients is not unrealistically slow. It could be a slowloris attack.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.