0

Windows Server 2008 R2 Datacenter Edition 64 bit

Using AWS EC2 m1.xlarge instance

15GB RAM , 4 vCPUs Intel Xeon

Problem: crashing every 10 secs / every minute / sometimes 1 hour / randomly

WINNT MPM Setting:

# WinNT MPM
# ThreadsPerChild: constant number of worker threads in the server process
# MaxConnectionsPerChild: maximum number of connections a server process serves
<IfModule mpm_winnt_module>
    ThreadStackSize   8388608
    ThreadsPerChild     500
    MaxRequestsPerChild    0
   SendBufferSize    32768
</IfModule>

Apache Server:

Server Version: Apache/2.4.29 (Win64) OpenSSL/1.0.2n

Server MPM: WinNT

Apache Lounge VC11 Server built: Nov 6 2017 11:17:28

Using PHP 5.5.38 with mod_php

Content in mod_status:

Server uptime: 1 hour 3 minutes 30 seconds
Server load: -1.00 -1.00 -1.00
Total accesses: 132819 - Total Traffic: 418.0 MB
34.9 requests/sec - 112.3 kB/second - 3299 B/request
115 requests currently being processed, 385 idle workers

Content in apache error.log , constant crashing every few mins

[Thu Jan 09 14:06:41.869757 2020] [mpm_winnt:notice] [pid 7408:tid 260] AH00428: Parent: child process 3872 exited with status 255 -- Restarting.
[Thu Jan 09 14:06:42.821339 2020] [mpm_winnt:notice] [pid 7408:tid 260] AH00455: Apache/2.4.29 (Win64) OpenSSL/1.0.2n configured -- resuming normal operations
[Thu Jan 09 14:06:42.821339 2020] [mpm_winnt:notice] [pid 7408:tid 260] AH00456: Apache Lounge VC11 Server built: Nov  6 2017 11:17:28
[Thu Jan 09 14:06:42.821339 2020] [core:notice] [pid 7408:tid 260] AH00094: Command line: 'C:\\Apache24\\bin\\httpd.exe -d C:/Apache24'
[Thu Jan 09 14:06:42.821339 2020] [mpm_winnt:notice] [pid 7408:tid 260] AH00418: Parent: Created child process 2588
[Thu Jan 09 14:06:44.100514 2020] [mpm_winnt:notice] [pid 2588:tid 156] AH00354: Child: Starting 500 worker threads.
[Thu Jan 09 14:06:47.470049 2020] [mpm_winnt:notice] [pid 7408:tid 260] AH00428: Parent: child process 2588 exited with status 255 -- Restarting.
[Thu Jan 09 14:06:48.265634 2020] [mpm_winnt:notice] [pid 7408:tid 260] AH00455: Apache/2.4.29 (Win64) OpenSSL/1.0.2n configured -- resuming normal operations
[Thu Jan 09 14:06:48.265634 2020] [mpm_winnt:notice] [pid 7408:tid 260] AH00456: Apache Lounge VC11 Server built: Nov  6 2017 11:17:28
[Thu Jan 09 14:06:48.265634 2020] [core:notice] [pid 7408:tid 260] AH00094: Command line: 'C:\\Apache24\\bin\\httpd.exe -d C:/Apache24'
[Thu Jan 09 14:06:48.265634 2020] [mpm_winnt:notice] [pid 7408:tid 260] AH00418: Parent: Created child process 5876
[Thu Jan 09 14:06:49.279615 2020] [mpm_winnt:notice] [pid 5876:tid 156] AH00354: Child: Starting 500 worker threads.

Content in Windows Event Viewer

Faulting application name: httpd.exe, version: 2.4.29.0, time stamp: 0x5a00349c
Faulting module name: ntdll.dll, version: 6.1.7601.23677, time stamp: 0x589c99e1
Exception code: 0xc0000005
Fault offset: 0x000000000004f23c
Faulting process id: 0x1ff4
Faulting application start time: 0x01d5c6b2a8435c36
Faulting application path: C:\Apache24\bin\httpd.exe
Faulting module path: C:\Windows\SYSTEM32\ntdll.dll
Report Id: 1b83564d-32a6-11ea-a95b-22000a1f2f53

Question: Why is it using C:\Windows\SYSTEM32\ntdll.dll when it is 64 bit apache? Could this be the issue?

What I tried to solve this issue:

  1. Tried modify MPM setting ThreadsPerChild from 1500 down to 200, still crashing

  2. Tested setting PHP.ini memory limit from 500MB up to 2G, still crashing

  3. Updated Apache to latest 2.4.38 , still got the issue

Is there anyway to debug this issue? Find out which module / code is causing this?

From what I searched, error code 0xc0000005, also known as Access Violation error.

LATEST UPDATE:

Upgraded PHP to version 5.6.40 (Last updated on Jan 2019) and observed a lot less crashes. Around 5-10 times in 24 hours within last 4 days.

The crashes are no longer with faulting module ntdll.dll, but with php5ts.dll and LIBLEAY32.dll (ssl related), thus no longer related to this thread.

Improve this question

1 Answer 1

Reset to default
1

You do know that your operating system will be unsupported by Microsoft in 4 days, right? And your php version has been unsupported for years already. Just so you know.

But for your problem, try disabling php for a bit just to see if it still crashes after that. There are some issues with apache for windows and running php as a module so maybe that's the problem. If it still crashes after that then at least you've ruled it out as the cause.

Improve this answer
3
  • Yes i am aware of that but we have been using it for many years, until recently it starts to crash like that. I tried running apache bench to send 150 concurrent request to a static image file (without php) it crashes too. Maybe it wasn't PHP.
    – user3162662
    Jan 11, 2020 at 1:25
  • 1
    Ah, you never mentioned it had been running for years ;) That does make it weird. I figured it was a fresh install for really old code.. If it has been running flawlessly for years, did this perhaps start after a windows update?
    – Xzenor
    Jan 11, 2020 at 12:15
  • There is no update that I am aware of. Because the server never restart. The only changes are the PHP codes, which is always updated because it is an ongoing platform. Might be some codes causing this, but it is too hard to debug, the project is too huge. And there is no PHP error, making it even harder to debug. I used Procdump to dump the memory during crash here but I have no idea how to debug that : stackoverflow.com/questions/59704971/…
    – user3162662
    Jan 13, 2020 at 17:01

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .