2

We are running Drupal 6.26, PHP 5.3 on Apache 2.2 mysql5.5. The Prod webserver has loadbalancer running with two PROD appservers. The Prod Webserver went down with httpd.exe using 100% CPU.

Below is the extract of the crash dump. This is the thread where Apache hangs.

.  0  Id: 126c.c20 Suspend: 0 Teb: 7efdd000 Unfrozen
ChildEBP RetAddr  
000cfd60 765615e9 ntdll!ZwWaitForMultipleObjects+0x15
000cfdfc 762319fc KERNELBASE!WaitForMultipleObjectsEx+0x100
000cfe44 762341d8 kernel32!WaitForMultipleObjectsExImplementation+0xe0
000cfe60 6ff1dc10 kernel32!WaitForMultipleObjects+0x18
WARNING: Stack unwind information not available. Following frames may be wrong.
000cfe9c 6ff20655 libhttpd!ap_regkey_value_remove+0x710
000cfec4 004017c7 libhttpd!ap_mpm_run+0x1c5
000cff14 00401f73 httpd+0x17c7
000cff88 7623336a httpd+0x1f73
000cff94 776a9f72 kernel32!BaseThreadInitThunk+0xe
000cffd4 776a9f45 ntdll!__RtlUserThreadStart+0x70
000cffec 00000000 ntdll!_RtlUserThreadStart+0x1b
Michael Hampton
  • 244,070
  • 43
  • 506
  • 972
Bhavna
  • 21
  • 2
  • 1
    Has this only happened once? If it happened more than once, was the crash dump the same, or different? – Michael Hampton Feb 18 '14 at 16:56
  • It has happened many times and everytime we have to restart the server. Though, we got hold of the crash dump only this time. – Bhavna Feb 18 '14 at 17:02
  • You should be able to run the crash dump through DebugView to figure out what caused the hang. It smells of hardware. – Nathan C Feb 18 '14 at 17:08
  • FWIW, Drupal 6 is is not really expected to work properly on PHP 5.3. See https://www.drupal.org/requirements – coleopterist Aug 12 '14 at 19:21

1 Answers1

0

That sure looks like locking (that is, locking in the mutual exclusion sense) code, but it's hard to really know for sure. As the warning states, the only way to ensure you're getting correct information is to have Apache running under a debugger when the failure occurs. This same method will also ensure that you are able to pull other relevant information about the system's state.

Of course, the practical solution may be to simply throw some monkeys at some typewriters and see if you can't thereby dance out of the problem without having to resort to something as painful as trace-debugging Apache on Windows. For example, you could try a different distribution of the WAMP stack, or switch operating systems on the servers (if that's practical). Ultimately, this is a very tough nut to crack, so both routes—finding and solving the problem, or doing an end-run around it—deserve full consideration.

For what it's worth, on Linux, I've tended to find these issues crop up the most around opcache caches, optimizers, and (de)obfuscators. If you are running such code—e.g. IonCube, ZendGuard, APC, the Windows Cache Extension for PHP, etc.—consider turning it off for an hour or two and see if the problems disappear.

BMDan
  • 7,249
  • 2
  • 23
  • 34