In the last few months we're experiencing a major issue in our production php servers:
The httpd running on our linux servers hangs and not responding to new requests (up to request timeout) until we manually restart it.
We've documented this issue quite a bit, and I can point out a few major investigation directions we want to follow, but before starting the research i wanted to ask the community if someone else might be experiencing a similar problem.
we are running 4 instances of linux servers, with version:
linux- 2.6.32-71, red had 4.4.4-13
and httpd versions: 2.2.15
the php versions we are running are 5.5.15 and 5.5.10 (2 servers on each version).
we have detected a few suspicious errors in the logs, but notably couldnt find a specific error near the crashes :
Servers seems busy, (you may need to increase StartServers, or Min/MaxSpareServers) spawning X children, there are Y idle and Z total children
mod_wsgi Initializing Python
- mod_wsgi Terminating Python
- mod_wsgi Cleanup interpeter: ''
- mod_wsgi: Destroying interpeters
- KeyError(NUMBER) in ignored
this is a php server, why is it throwing python errors?
SIGHUP received. Attempting to restart - * glibc detected * /usr/bin/httpd: corrupted double-linked list: ADDRESS ***
Connection reset by peer: core_output_filter: writing data to network
- We noticed that "/var/log/httpd/error_log" and "/var/log/httpd/access_log" removed every time the error occur. We are having a really hard time with this subject, and would really appriciate it if someone can help shed some light on this matter.
Thank you very much!