0

We have an Asp.Net application (Framework 3.5) hosted from 3 servers using a load-balancer. Recently we started getting crashes in all the three production servers.

Faulting application name: w3wp.exe, version: 7.5.7601.17514, time stamp: 0x4ce7afa2 Faulting module name: KERNELBASE.dll, version: 6.1.7601.18847, time stamp: 0x554d7df1 Exception code: 0xe053534f Fault offset: 0x000000000000b3dd Faulting process id: 0x%9 Faulting application start time: 0x%10 Faulting application path: %11 Faulting module path: %12 Report Id: %13

I generated a crash dump using windbg as follows.

Crash - specific process - w3wp - Added exception "e053534f"

I tried to anlayze the dump but I am not getting anywhere. An extract from the dump is given below.

0:044>  .foreach (ex {!dumpheap -type Exception -short}){.echo      "********************************";!pe ${ex} }
********************************
Not a valid exception object
********************************
Not a valid exception object
********************************
Not a valid exception object
********************************
Not a valid exception object
********************************
Exception object: 00000000ff6fd900
Exception type: System.NotSupportedException
Message: Specified method is not supported.
InnerException: <none>
StackTrace (generated):
<none>
StackTraceString: <none>
HResult: 80131515
********************************
Not a valid exception object
********************************
Exception object: 000000013f3100b0
Exception type: System.OutOfMemoryException
Message: <none>
InnerException: <none>
StackTrace (generated):
<none>
StackTraceString: <none>
HResult: 8007000e
********************************
Exception object: 000000013f310138
Exception type: System.StackOverflowException
Message: <none>
InnerException: <none>
StackTrace (generated):
<none>
StackTraceString: <none>
HResult: 800703e9
********************************
Exception object: 000000013f3101c0
Exception type: System.ExecutionEngineException
Message: <none>
InnerException: <none>
StackTrace (generated):
<none>
StackTraceString: <none>
HResult: 80131506
********************************
Exception object: 000000013f310248
Exception type: System.Threading.ThreadAbortException
Message: <none>
InnerException: <none>
StackTrace (generated):
<none>
StackTraceString: <none>
HResult: 80131530
********************************
Exception object: 000000013f3102d0
Exception type: System.Threading.ThreadAbortException
Message: <none>
InnerException: <none>
StackTrace (generated):
<none>
StackTraceString: <none>
HResult: 80131530
********************************
Not a valid exception object
********************************
Not a valid exception object
********************************
Not a valid exception object
********************************
Not a valid exception object
********************************
Exception object: 000000014265d7b0
Exception type: System.ObjectDisposedException
Message: Cannot access a disposed object.
InnerException: <none>
StackTrace (generated):
<none>
StackTraceString: <none>
HResult: 80131622
********************************
Not a valid exception object
********************************
Not a valid exception object
********************************
Not a valid exception object
********************************
Not a valid exception object
********************************
Not a valid exception object
********************************
Not a valid exception object
********************************
Not a valid exception object
********************************
Not a valid exception object
********************************
Not a valid exception object
********************************
Not a valid exception object
********************************
Not a valid exception object
********************************
Not a valid exception object
********************************
Unknown option: ------------------------------
0:044> !threads
ThreadCount: 34
UnstartedThread: 0
BackgroundThread: 34
PendingThread: 0
DeadThread: 0
Hosted Runtime: no
                                              PreEmptive                                                Lock
       ID OSID        ThreadOBJ     State   GC     GC Alloc Context                  Domain           Count APT Exception
   4    1 107c 0000000002549640      8220 Enabled  0000000000000000:0000000000000000 0000000002544870     0 Ukn
  21    2  c28 000000000255c760      b220 Enabled  0000000181becd38:0000000181bee858 0000000002544870     0 MTA (Finalizer)
  15    3 1b28 0000000002599fd0   880a220 Enabled  000000018416e0d8:000000018416ff60 0000000002544870     0 MTA (Threadpool Completion Port)
  22    4  86c 000000000259c760    80a220 Enabled  0000000000000000:0000000000000000 0000000002544870     0 MTA (Threadpool Completion Port)
  23    5 1700 000000000259da10      1220 Enabled  0000000000000000:0000000000000000 0000000002544870     0 Ukn
  24    8 1d50 0000000005035b50   180b220 Enabled  0000000000000000:0000000000000000 0000000002544870     0 MTA (Threadpool Worker)
  27    b  6e8 00000000051e77a0   200b220 Enabled  0000000000000000:0000000000000000 000000000259e4c0     0 MTA
   3   12 1d5c 00000000025b4460       220 Enabled  0000000000000000:0000000000000000 0000000002544870     0 Ukn
  25   14  37c 00000000025b5ba0       220 Enabled  0000000000000000:0000000000000000 0000000002544870     0 Ukn
   6   18 1fa4 0000000004ed5bc0       220 Enabled  0000000000000000:0000000000000000 0000000002544870     0 Ukn
   5   19 1564 0000000004ed6190       220 Enabled  0000000000000000:0000000000000000 0000000002544870     0 Ukn
  30    7 18b4 0000000004ed55f0   380b220 Enabled  0000000000000000:0000000000000000 000000000259e4c0     2 MTA (Threadpool Worker)
  31    9 1c64 00000000025b3e90   180b220 Enabled  0000000000000000:0000000000000000 0000000002544870     0 MTA (Threadpool Worker)
  32    6 190c 0000000004ed78d0   180b220 Enabled  0000000000000000:0000000000000000 0000000002544870     0 MTA (Threadpool Worker)
  33    f 1a60 0000000004ed6d30   200b220 Enabled  000000014265f930:00000001426618b8 000000000259e4c0     1 MTA
  34   11 1484 000000000fcb4d90       220 Enabled  0000000000000000:0000000000000000 0000000002544870     0 Ukn
  35    c 19b4 000000000fcb3080       220 Enabled  0000000000000000:0000000000000000 0000000002544870     0 Ukn
  37   25 11a8 000000000d6da270   180b220 Enabled  000000010516b2c8:000000010516c230 0000000002544870     0 MTA (Threadpool Worker)
  38   2e 1b08 000000000d795e60   180b220 Enabled  0000000000000000:0000000000000000 0000000002544870     0 MTA (Threadpool Worker)
  39   38 1d0c 000000000db5a950   180b220 Enabled  00000001430b3ab0:00000001430b5a20 0000000002544870     0 MTA (Threadpool Worker)
  40   2d 1c8c 000000000d795890   180b220 Enabled  000000018416ca38:000000018416df60 0000000002544870     0 MTA (Threadpool Worker)
  41   34 1608 000000000d691c10   180b220 Enabled  0000000104fcd8b0:0000000104fcdc18 0000000002544870     0 MTA (Threadpool Worker)
  36   2b 1ba8 000000000db5a380       220 Enabled  0000000000000000:0000000000000000 0000000002544870     0 Ukn
  43   35 14a0 000000000d6921e0   180b220 Enabled  0000000000000000:0000000000000000 0000000002544870     0 MTA (Threadpool Worker)
  44   1a 1dc0 000000000fcb3650   180b220 Disabled 0000000143390598:0000000143391a58 000000000259e4c0     1 MTA (Threadpool Worker)
  45   37  b08 000000000fcb5f00   180b220 Enabled  0000000000000000:0000000000000000 0000000002544870     0 MTA (Threadpool Worker)
  46   13 11ac 000000000fcb5360       220 Enabled  0000000000000000:0000000000000000 0000000002544870     0 Ukn
  47   32  e5c 000000000fcb5930   180b220 Enabled  0000000000000000:0000000000000000 0000000002544870     0 MTA (Threadpool Worker)
  48   17  cdc 000000000fcb47c0   180b220 Enabled  0000000000000000:0000000000000000 0000000002544870     0 MTA (Threadpool Worker)
  49   24  ba8 000000000d6d9ca0   180b220 Enabled  0000000000000000:0000000000000000 0000000002544870     0 MTA (Threadpool Worker)
  42    a 1ac0 000000000fcb41f0       220 Enabled  0000000000000000:0000000000000000 0000000002544870     0 Ukn
  50   21 18cc 000000000fcb3c20   180b220 Enabled  00000001c6d2cd00:00000001c6d2dfa0 0000000002544870     0 MTA (Threadpool Worker)
  51   31 1a58 000000000db5af20   180b220 Enabled  0000000000000000:0000000000000000 0000000002544870     0 MTA (Threadpool Worker)
  52   1c 154c 000000000db5b4f0   180b220 Enabled  0000000000000000:0000000000000000 0000000002544870     0 MTA (Threadpool Worker)
0:044> !pe
There is no current managed exception on this thread
0:044> ~*e !pe
The current thread is unmanaged
The current thread is unmanaged
The current thread is unmanaged
There is no current managed exception on this thread
There is no current managed exception on this thread
There is no current managed exception on this thread
There is no current managed exception on this thread
The current thread is unmanaged
The current thread is unmanaged
The current thread is unmanaged
The current thread is unmanaged
The current thread is unmanaged
The current thread is unmanaged
The current thread is unmanaged
The current thread is unmanaged
There is no current managed exception on this thread
The current thread is unmanaged
The current thread is unmanaged
The current thread is unmanaged
The current thread is unmanaged
The current thread is unmanaged
There is no current managed exception on this thread
There is no current managed exception on this thread
There is no current managed exception on this thread
There is no current managed exception on this thread
There is no current managed exception on this thread
The current thread is unmanaged
There is no current managed exception on this thread
The current thread is unmanaged
The current thread is unmanaged
There is no current managed exception on this thread
There is no current managed exception on this thread
There is no current managed exception on this thread
There is no current managed exception on this thread
There is no current managed exception on this thread
There is no current managed exception on this thread
There is no current managed exception on this thread
There is no current managed exception on this thread
There is no current managed exception on this thread
There is no current managed exception on this thread
There is no current managed exception on this thread
There is no current managed exception on this thread
There is no current managed exception on this thread
There is no current managed exception on this thread
There is no current managed exception on this thread
There is no current managed exception on this thread
There is no current managed exception on this thread
There is no current managed exception on this thread
There is no current managed exception on this thread
There is no current managed exception on this thread
There is no current managed exception on this thread
There is no current managed exception on this thread
There is no current managed exception on this thread
0:044> !CLRStack -p -l
OS Thread Id: 0x1dc0 (44)
(!clrstack processes a max of 1000 stack frames)
Child-SP         RetAddr          Call Site
000000000f2d7280 000007fef920c365 System.Enum.System.IConvertible.ToInt64(System.IFormatProvider)
    PARAMETERS:
        this = <no data>
        provider = <no data>

000000000f2d72c0 000007fef921ae7d System.Convert.ToInt64(System.Object, System.IFormatProvider)
    PARAMETERS:
        value = <no data>
        provider = <no data>

000000000f2d72f0 000007fef9223aaf System.Enum.ToUInt64(System.Object)
    PARAMETERS:
        value = <no data>
    LOCALS:
        <no data>
        <no data>
        <no data>

000000000f2d7330 000007ff00425ead System.Enum.IsDefined(System.Type, System.Object)
    PARAMETERS:
        enumType = <no data>
        value = <no data>
    LOCALS:
        <no data>
        <no data>
        <no data>
        <CLR reg> = 0x0000000140d2b1b0
        <no data>
        <no data>
        <no data>
        <no data>
        <no data>

000000000f2d7390 000007ff00425f17 Microsoft.Practices.ObjectBuilder.Locator.Get(System.Object, Microsoft.Practices.ObjectBuilder.SearchMode)
    PARAMETERS:
        this = <no data>
        key = <no data>
        options = <no data>

000000000f2d73f0 000007ff00425df0 Microsoft.Practices.ObjectBuilder.Locator.Get(System.Object, Microsoft.Practices.ObjectBuilder.SearchMode)
    PARAMETERS:
        this = <no data>
        key = <no data>
        options = <no data>

000000000f2d7450 000007ff0042f7cf Microsoft.Practices.ObjectBuilder.ReadableLocator.Get[[System.__Canon, mscorlib]](System.Object)
    PARAMETERS:
        this = <no data>
        key = <no data>

000000000f2d7480 000007ff00429206 Microsoft.Practices.CompositeWeb.BuilderStrategies.SessionStateBindingStrategy.BuildUp(Microsoft.Practices.ObjectBuilder.IBuilderContext, System.Type, System.Object, System.String)
    PARAMETERS:
        this = 0x00000001c1a44a20
        context = 0x000000014331e7f0
        typeToBuild = 0x000000017f3d0e08
        existing = 0x0000000143380460
        idToBuild = 0x00000001433193d0
    LOCALS:
        0x000000000f2d74b0 = 0x0000000000000000
        0x000000000f2d74b8 = 0x0000000000000000
        0x000000000f2d74c0 = 0x0000000000000000
        0x000000000f2d74c8 = 0x0000000000000000
        0x000000000f2d74d0 = 0x0000000000000000
        0x000000000f2d74d8 = 0x0000000000000000
        0x000000000f2d74e0 = 0x0000000000000000

000000000f2d7560 000007ff0042f96e Microsoft.Practices.ObjectBuilder.BuilderStrategy.BuildUp(Microsoft.Practices.ObjectBuilder.IBuilderContext, System.Type, System.Object, System.String)
    PARAMETERS:
        this = <no data>
        context = <no data>
        typeToBuild = <no data>
        existing = <no data>
        idToBuild = <no data>
    LOCALS:
        <CLR reg> = 0x000000000f2d68b0

000000000f2d75b0 000007ff00429206 Microsoft.Practices.CompositeWeb.BuilderStrategies.SessionStateBindingStrategy.BuildUp(Microsoft.Practices.ObjectBuilder.IBuilderContext, System.Type, System.Object, System.String)
    PARAMETERS:
        this = 0x0000000180c82190
        context = 0x000000014331e7f0
        typeToBuild = 0x000000017f3d0e08
        existing = 0x0000000143380460
        idToBuild = 0x00000001433193d0
    LOCALS:
        0x000000000f2d75e0 = 0x000000013f34a2c0
        0x000000000f2d75e8 = 0x00000001433904d0
        0x000000000f2d75f0 = 0x0000000000000000
        0x000000000f2d75f8 = 0x0000000000000000
        0x000000000f2d7600 = 0x0000000000000000
        0x000000000f2d7608 = 0x0000000143390510
        0x000000000f2d7610 = 0x0000000000000000

000000000f2d7690 000007ff0042f96e Microsoft.Practices.ObjectBuilder.BuilderStrategy.BuildUp(Microsoft.Practices.ObjectBuilder.IBuilderContext, System.Type, System.Object, System.String)
    PARAMETERS:
        this = <no data>
        context = <no data>
        typeToBuild = <no data>
        existing = <no data>
        idToBuild = <no data>
    LOCALS:
        <CLR reg> = 0x000000000f2d68b0

000000000f2d76e0 000007ff00429206 Microsoft.Practices.CompositeWeb.BuilderStrategies.SessionStateBindingStrategy.BuildUp(Microsoft.Practices.ObjectBuilder.IBuilderContext, System.Type, System.Object, System.String)
    PARAMETERS:
        this = 0x00000001c0e00448
        context = 0x000000014331e7f0
        typeToBuild = 0x000000017f3d0e08
        existing = 0x0000000143380460
        idToBuild = 0x00000001433193d0
    LOCALS:
        0x000000000f2d7710 = 0x000000013f34a2c0
        0x000000000f2d7718 = 0x00000001433903f0
        0x000000000f2d7720 = 0x0000000000000000
        0x000000000f2d7728 = 0x0000000000000000
        0x000000000f2d7730 = 0x0000000000000000
        0x000000000f2d7738 = 0x0000000143390430
        0x000000000f2d7740 = 0x0000000000000000

000000000f2d77c0 000007ff0042f96e Microsoft.Practices.ObjectBuilder.BuilderStrategy.BuildUp(Microsoft.Practices.ObjectBuilder.IBuilderContext, System.Type, System.Object, System.String)
    PARAMETERS:
        this = <no data>
        context = <no data>
        typeToBuild = <no data>
        existing = <no data>
        idToBuild = <no data>
    LOCALS:
        <CLR reg> = 0x000000000f2d68b0

000000000f2d7810 000007ff00429206 Microsoft.Practices.CompositeWeb.BuilderStrategies.SessionStateBindingStrategy.BuildUp(Microsoft.Practices.ObjectBuilder.IBuilderContext, System.Type, System.Object, System.String)
    PARAMETERS:
        this = 0x0000000140cd6438
        context = 0x000000014331e7f0
        typeToBuild = 0x000000017f3d0e08
        existing = 0x0000000143380460
        idToBuild = 0x00000001433193d0
    LOCALS:
        0x000000000f2d7840 = 0x000000013f34a2c0
        0x000000000f2d7848 = 0x0000000143390310
        0x000000000f2d7850 = 0x0000000000000000
        0x000000000f2d7858 = 0x0000000000000000
        0x000000000f2d7860 = 0x0000000000000000
        0x000000000f2d7868 = 0x0000000143390350
        0x000000000f2d7870 = 0x0000000000000000

000000000f2d78f0 000007ff0042f96e Microsoft.Practices.ObjectBuilder.BuilderStrategy.BuildUp(Microsoft.Practices.ObjectBuilder.IBuilderContext, System.Type, System.Object, System.String)
    PARAMETERS:
        this = <no data>
        context = <no data>
        typeToBuild = <no data>
        existing = <no data>
        idToBuild = <no data>
    LOCALS:
        <CLR reg> = 0x000000000f2d68b0

000000000f2d7940 000007ff00429206 Microsoft.Practices.CompositeWeb.BuilderStrategies.SessionStateBindingStrategy.BuildUp(Microsoft.Practices.ObjectBuilder.IBuilderContext, System.Type, System.Object, System.String)
    PARAMETERS:
        this = 0x00000001c0deed28
        context = 0x000000014331e7f0
        typeToBuild = 0x000000017f3d0e08
        existing = 0x0000000143380460
        idToBuild = 0x00000001433193d0
    LOCALS:
        0x000000000f2d7970 = 0x000000013f34a2c0
        0x000000000f2d7978 = 0x0000000143390230
        0x000000000f2d7980 = 0x0000000000000000
        0x000000000f2d7988 = 0x0000000000000000
        0x000000000f2d7990 = 0x0000000000000000
        0x000000000f2d7998 = 0x0000000143390270
        0x000000000f2d79a0 = 0x0000000000000000

I am not able to locate where this is happening. Looks like the dump is not showing any details about the page or function where the error occurred. Can some one point me to the right way of analyzing this?

Thanks, Anup

Anup
  • 9
  • 1
  • 6

1 Answers1

0

0xe053534f is a stack overflow exception, but not a managed one, so analyzing with managed tools like SOS does not help.

If you don't have much WinDbg experience, you may try DebugDiag; Version 2.1 is really good and often helps finding the problem instantly.

In WinDbg, start with the following commands:

.symfix c:\symbols
.reload /f
.exr -1

Should give the exception code, just to cross check that the dump contains the correct information (Stack Overflow Exception)

~#s

Select the thread where the exception happened.

kbn

Show the native callstack. Since this seems to be mixed mode, you can also try

!dumpstack

or 's

!mk

to see what happened before on the .NET side.

Thomas Weller
  • 55,411
  • 20
  • 125
  • 222
  • Hi Thanks for the details. I tried DebugDiag analysis but did not find anything useful there. I get the following - "In w3wp__MPulse7__PID__5540__Date__08_20_2015__Time_02_39_34PM__991__First Chance Exception 0XE053534F.dmp the assembly instruction at KERNELBASE!RaiseException+39 in C:\Windows\System32\KERNELBASE.dll from Microsoft Corporation has caused an unknown exception (0xe053534f) on thread 44". When I tried thread 44, it showed the same as posted in my previous post. I shall try windbg with the commands that you have specified. Thanks, Anup – Anup Aug 20 '15 at 22:03
  • Hi, I tried this and !mk is giving me a message which says "No export mk found" – Anup Aug 20 '15 at 22:29
  • @Anup: In order to use `!mk`, you have to download SOSEX, load the DLL in WinDbg `.load c:\pathto\sosex.dll`. – Thomas Weller Aug 22 '15 at 12:16
  • @Anup: BTW: a first chance exception does not necessarily crash the process. – Thomas Weller Aug 22 '15 at 12:17