0

We have a number of WCF services hosted in IIS with a WPF front end. Occasionally IIS will crash, with what looks to be a serialisation error but unsure where or what object is failing. We suspect its the serialisation of an entity framework object collection (with navigation properties etc), but the object in question serialises fine when tested internally. IIS crashes maybe once/twice a day onsite. The object in question is used heavily by end users, which leads me to believe if this is a genuine serialisation fault then we would see serialisation over WCF fail every time? Could it be failing only on certain occasions, or could something else be causing IIS to crash like this? We're finding this difficult to track down.

There is an entry in the event log from source 'WAS'

A process serving application pool 'xxx' suffered a fatal communication error with the Windows Process Activation Service. The process ID was '1016'. The data field contains the error number.

This is the memory dump/ stack trace captured using DebugDiag tool.

The following threads in w3wp__appPoolName__PID__2932__Date__09_29_2016__Time_01_00_26PM__964__Ntdll!ZwTerminateProcess.dmp are calling the TerminateProcess function to kill the process

( 39 )

2,13% of threads blocked (1 threads)

Please look at the callstack of the thread to see what it is doing which is ending up in a call to TerminateProcess function

Thread 39 - System ID 4564

Entry point clr!GetPrivateContextsPerfCounters+16f00 Create time
29.09.2016 12:59:25 Time spent in user mode 0 Days 00:00:08.125 Time spent in kernel mode 0 Days 00:00:00.000

This thread is calling the TerminateProcess function to kill the process

The thread has evidence of .net exceptions on the stack. Check the Previous .NET Exceptions Report (Exceptions in all .NET Thread Stacks) to view more details of the associated exception

.NET Call Stack

[[FaultingExceptionFrame]] System_Runtime_Serialization_ni!System.Xml.XmlBaseWriter.WriteStartElement(System.String, System.Xml.XmlDictionaryString, System.Xml.XmlDictionaryString)+3b System_Runtime_Serialization_ni!System.Runtime.Serialization.XmlWriterDelegator.WriteStartElement(System.String, System.Xml.XmlDictionaryString, System.Xml.XmlDictionaryString)+23 DynamicClass.WriteEntityNameToXml(System.Runtime.Serialization.XmlWriterDelegator, System.Object, System.Runtime.Serialization.XmlObjectSerializerWriteContext, System.Runtime.Serialization.ClassDataContract)+14b9 System_Runtime_Serialization_ni!System.Runtime.Serialization.ClassDataContract.WriteXmlValue(System.Runtime.Serialization.XmlWriterDelegator, System.Object, System.Runtime.Serialization.XmlObjectSerializerWriteContext)+31 System_Runtime_Serialization_ni!System.Runtime.Serialization.XmlObjectSerializerWriteContext.SerializeAndVerifyType(System.Runtime.Serialization.DataContract, System.Runtime.Serialization.XmlWriterDelegator, System.Object, Boolean, System.RuntimeTypeHandle, System.Type)+eb System_Runtime_Serialization_ni!System.Runtime.Serialization.XmlObjectSerializerWriteContext.SerializeWithXsiType(System.Runtime.Serialization.XmlWriterDelegator, System.Object, System.RuntimeTypeHandle, System.Type, Int32, System.RuntimeTypeHandle, System.Type)+309 System_Runtime_Serialization_ni!System.Runtime.Serialization.XmlObjectSerializerWriteContext.InternalSerialize(System.Runtime.Serialization.XmlWriterDelegator, System.Object, Boolean, Boolean, Int32, System.RuntimeTypeHandle)+247897 System_Runtime_Serialization_ni!System.Runtime.Serialization.XmlObjectSerializerWriteContext.InternalSerializeReference(System.Runtime.Serialization.XmlWriterDelegator, System.Object, Boolean, Boolean, Int32, System.RuntimeTypeHandle)+6f DynamicClass.WriteObjectListToXml(System.Runtime.Serialization.XmlWriterDelegator, System.Object, System.Runtime.Serialization.XmlObjectSerializerWriteContext, System.Runtime.Serialization.CollectionDataContract)+1aa System_Runtime_Serialization_ni!System.Runtime.Serialization.CollectionDataContract.WriteXmlValue(System.Runtime.Serialization.XmlWriterDelegator, System.Object, System.Runtime.Serialization.XmlObjectSerializerWriteContext)+41 System_Runtime_Serialization_ni!System.Runtime.Serialization.XmlObjectSerializerWriteContext.InternalSerialize(System.Runtime.Serialization.XmlWriterDelegator, System.Object, Boolean, Boolean, Int32, System.RuntimeTypeHandle)+6f System_Runtime_Serialization_ni!System.Runtime.Serialization.XmlObjectSerializerWriteContext.InternalSerializeReference(System.Runtime.Serialization.XmlWriterDelegator, System.Object, Boolean, Boolean, Int32, System.RuntimeTypeHandle)+6f DynamicClass.WriteKeyValueOfstringObjectListtGI2ndrTToXml(System.Runtime.Serialization.XmlWriterDelegator, System.Object, System.Runtime.Serialization.XmlObjectSerializerWriteContext, System.Runtime.Serialization.ClassDataContract)+176 System_Runtime_Serialization_ni!System.Runtime.Serialization.ClassDataContract.WriteXmlValue(System.Runtime.Serialization.XmlWriterDelegator, System.Object, System.Runtime.Serialization.XmlObjectSerializerWriteContext)+31 DynamicClass.WriteObjectsRemovedFromCollectionPropertiesToXml(System.Runtime.Serialization.XmlWriterDelegator, System.Object, System.Runtime.Serialization.XmlObjectSerializerWriteContext, System.Runtime.Serialization.CollectionDataContract)+1a1 System_Runtime_Serialization_ni!System.Runtime.Serialization.CollectionDataContract.WriteXmlValue(System.Runtime.Serialization.XmlWriterDelegator, System.Object, System.Runtime.Serialization.XmlObjectSerializerWriteContext)+41 System_Runtime_Serialization_ni!System.Runtime.Serialization.XmlObjectSerializerWriteContext.InternalSerialize(System.Runtime.Serialization.XmlWriterDelegator, System.Object, Boolean, Boolean, Int32, System.RuntimeTypeHandle)+6f System_Runtime_Serialization_ni!System.Runtime.Serialization.XmlObjectSerializerWriteContext.InternalSerializeReference(System.Runtime.Serialization.XmlWriterDelegator, System.Object, Boolean, Boolean, Int32, System.RuntimeTypeHandle)+6f DynamicClass.WriteEntityChangeTrackerToXml(System.Runtime.Serialization.XmlWriterDelegator, System.Object, System.Runtime.Serialization.XmlObjectSerializerWriteContext, System.Runtime.Serialization.ClassDataContract)+31a System_Runtime_Serialization_ni!System.Runtime.Serialization.ClassDataContract.WriteXmlValue(System.Runtime.Serialization.XmlWriterDelegator, System.Object, System.Runtime.Serialization.XmlObjectSerializerWriteContext)+31 System_Runtime_Serialization_ni!System.Runtime.Serialization.XmlObjectSerializerWriteContext.InternalSerialize(System.Runtime.Serialization.XmlWriterDelegator, System.Object, Boolean, Boolean, Int32, System.RuntimeTypeHandle)+6f...

... serialisation continues like this, followed by...

...System_ServiceModel_ni!System.ServiceModel.Dispatcher.ChannelHandler.DispatchAndReleasePump(System.ServiceModel.Channels.RequestContext, Boolean, System.ServiceModel.OperationContext)+7d7 System_ServiceModel_ni!System.ServiceModel.Dispatcher.ChannelHandler.HandleRequest(System.ServiceModel.Channels.RequestContext, System.ServiceModel.OperationContext)+1ff System_ServiceModel_ni!System.ServiceModel.Dispatcher.ChannelHandler.AsyncMessagePump(System.IAsyncResult)+db System_ServiceModel_Internals_ni!System.Runtime.Fx+AsyncThunk.UnhandledExceptionFrame(System.IAsyncResult)+32 System_ServiceModel_Internals_ni!System.Runtime.AsyncResult.Complete(Boolean)+19e System_ServiceModel_ni!System.ServiceModel.Channels.TransportDuplexSessionChannel+TryReceiveAsyncResult.OnReceive(System.IAsyncResult)+11d System_ServiceModel_Internals_ni!System.Runtime.Fx+AsyncThunk.UnhandledExceptionFrame(System.IAsyncResult)+32 System_ServiceModel_Internals_ni!System.Runtime.AsyncResult.Complete(Boolean)+19e System_ServiceModel_ni!System.ServiceModel.Channels.SynchronizedMessageSource+ReceiveAsyncResult.OnReceiveComplete(System.Object)+82 System_ServiceModel_ni!System.ServiceModel.Channels.SessionConnectionReader.OnAsyncReadComplete(System.Object)+170 System_ServiceModel_ni!System.ServiceModel.Channels.SocketConnection.OnReceiveAsync(System.Object, System.Net.Sockets.SocketAsyncEventArgs)+fd System_ni!System.Net.Sockets.SocketAsyncEventArgs.FinishOperationSuccess(System.Net.Sockets.SocketError, Int32, System.Net.Sockets.SocketFlags)+bd5 System_ni!System.Net.Sockets.SocketAsyncEventArgs.CompletionPortCallback(UInt32, UInt32, System.Threading.NativeOverlapped*)+40 mscorlib_ni!System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32, UInt32, System.Threading.NativeOverlapped*)+96 [[GCFrame]] [[DebuggerU2MCatchHandlerFrame]] [[ContextTransitionFrame]] [[DebuggerU2MCatchHandlerFrame]]

Full Stack Trace

System_Core_ni!System.Linq.Expressions.ConstantExpression.Make(System.Object, System.Type)+69 System.Data.Entity.Core.Objects.ELinq.ExpressionConverter+ConstantTranslator.TypedTranslate(System.Data.Entity.Core.Objects.ELinq.ExpressionConverter, System.Linq.Expressions.ConstantExpression)+12d 0x000000dc6cf0c5d0 clr!GetMetaDataPublicInterfaceFromInternal+3c494 System.Data.Entity.Core.Objects.ELinq.ExpressionConverter.TranslateExpression(System.Linq.Expressions.Expression)+ae System.Data.Entity.Core.Objects.ELinq.ExpressionConverter+MethodCallTranslator+ContainsTranslator.TranslateContains(System.Data.Entity.Core.Objects.ELinq.ExpressionConverter, System.Linq.Expressions.Expression, System.Linq.Expressions.Expression)+46 System.Data.Entity.Core.Objects.ELinq.ExpressionConverter+MethodCallTranslator.TypedTranslate(System.Data.Entity.Core.Objects.ELinq.ExpressionConverter, System.Linq.Expressions.MethodCallExpression)+d3 System.Data.Entity.Core.Objects.ELinq.ExpressionConverter+BinaryTranslator.TypedTranslate(System.Data.Entity.Core.Objects.ELinq.ExpressionConverter, System.Linq.Expressions.BinaryExpression)+132 System.Data.Entity.Core.Objects.ELinq.ExpressionConverter.TranslateExpression(System.Linq.Expressions.Expression)+ae System.Data.Entity.Core.Objects.ELinq.ExpressionConverter.TranslateLambda(System.Linq.Expressions.LambdaExpression, System.Data.Entity.Core.Common.CommandTrees.DbExpression)+7e System.Data.Entity.Core.Objects.ELinq.ExpressionConverter+MethodCallTranslator+OneLambdaTranslator.Translate(System.Data.Entity.Core.Objects.ELinq.ExpressionConverter, System.Linq.Expressions.MethodCallExpression, System.Data.Entity.Core.Common.CommandTrees.DbExpression ByRef, System.Data.Entity.Core.Common.CommandTrees.DbExpressionBinding ByRef, System.Data.Entity.Core.Common.CommandTrees.DbExpression ByRef)+eb System.Data.Entity.Core.Objects.ELinq.ExpressionConverter+MethodCallTranslator+OneLambdaTranslator.Translate(System.Data.Entity.Core.Objects.ELinq.ExpressionConverter, System.Linq.Expressions.MethodCallExpression)+3d System.Data.Entity.Core.Objects.ELinq.ExpressionConverter+MethodCallTranslator.TypedTranslate(System.Data.Entity.Core.Objects.ELinq.ExpressionConverter, System.Linq.Expressions.MethodCallExpression)+d3 System.Data.Entity.Core.Objects.ELinq.ExpressionConverter.TranslateExpression(System.Linq.Expressions.Expression)+ae System.Data.Entity.Core.Objects.ELinq.ExpressionConverter+MethodCallTranslator+SelectTranslator.Translate(System.Data.Entity.Core.Objects.ELinq.ExpressionConverter, System.Linq.Expressions.MethodCallExpression)+65 System.Data.Entity.Core.Objects.ELinq.ExpressionConverter+MethodCallTranslator.TypedTranslate(System.Data.Entity.Core.Objects.ELinq.ExpressionConverter, System.Linq.Expressions.MethodCallExpression)+d3 System.Data.Entity.Core.Objects.ELinq.ExpressionConverter.Convert()+a2 System.Data.Entity.Core.Objects.ELinq.ELinqQueryState.GetExecutionPlan(System.Nullable1)+128 System.Data.Entity.Core.Objects.ObjectQuery1+<>c__DisplayClass3[[System.__Canon, mscorlib]].<GetResults>b__2()+27 System.Data.Entity.Core.Objects.ObjectContext.ExecuteInTransaction[[System.__Canon, mscorlib]](System.Func1, System.Data.Entity.Infrastructure.IDbExecutionStrategy, Boolean, Boolean)+110 [[StubHelperFrame]] System.Data.Entity.Core.Objects.ObjectQuery1+<>c__DisplayClass3[[System.__Canon, mscorlib]].<GetResults>b__1()+ef System.Data.Entity.SqlServer.DefaultSqlExecutionStrategy.Execute[[System.__Canon, mscorlib]](System.Func1)+3b System.Data.Entity.Core.Objects.ObjectQuery1[[System.__Canon, mscorlib]].GetResults(System.Nullable1)+170 System.Data.Entity.Core.Objects.ObjectQuery1[[System.__Canon, mscorlib]].<System.Collections.Generic.IEnumerable<T>.GetEnumerator>b__0()+b System.Data.Entity.Internal.LazyEnumerator1[[System.__Canon, mscorlib]].MoveNext()+32 mscorlib_ni!System.Collections.Generic.List1[[System.__Canon, mscorlib]]..ctor(System.Collections.Generic.IEnumerable1)+218 System_Core_ni!System.Linq.Enumerable.ToList[System.__Canon, mscorlib]+50 **Repository.MethodName**(System.Collections.Generic.List1)+1626 ServiceName.MethodName(System.Collections.Generic.List`1)+18 DynamicClass.SyncInvokeMethodName(System.Object, System.Object[], System.Object[])+7a System_ServiceModel_ni!System.ServiceModel.Dispatcher.SyncMethodInvoker.Invoke(System.Object, System.Object[], System.Object[] ByRef)+6a4 System_ServiceModel_ni!System.ServiceModel.Dispatcher.DispatchOperationRuntime.InvokeBegin(System.ServiceModel.Dispatcher.MessageRpc ByRef)+505 System_ServiceModel_ni!System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage5(System.ServiceModel.Dispatcher.MessageRpc ByRef)+110 System_ServiceModel_ni!System.ServiceModel.Dispatcher.ImmutableDispatchRuntime.ProcessMessage31(System.ServiceModel.Dispatcher.MessageRpc ByRef)+17b System_ServiceModel_ni!System.ServiceModel.Dispatcher.MessageRpc.Process(Boolean)+17e System_ServiceModel_ni!System.ServiceModel.Dispatcher.ChannelHandler.DispatchAndReleasePump(System.ServiceModel.Channels.RequestContext, Boolean, System.ServiceModel.OperationContext)+7d7 System_ServiceModel_ni!System.ServiceModel.Dispatcher.ChannelHandler.HandleRequest(System.ServiceModel.Channels.RequestContext, System.ServiceModel.OperationContext)+1ff System_ServiceModel_ni!System.ServiceModel.Dispatcher.ChannelHandler.AsyncMessagePump(System.IAsyncResult)+db System_ServiceModel_Internals_ni!System.Runtime.Fx+AsyncThunk.UnhandledExceptionFrame(System.IAsyncResult)+32 System_ServiceModel_Internals_ni!System.Runtime.AsyncResult.Complete(Boolean)+19e System_ServiceModel_ni!System.ServiceModel.Channels.TransportDuplexSessionChannel+TryReceiveAsyncResult.OnReceive(System.IAsyncResult)+11d System_ServiceModel_Internals_ni!System.Runtime.Fx+AsyncThunk.UnhandledExceptionFrame(System.IAsyncResult)+32 System_ServiceModel_Internals_ni!System.Runtime.AsyncResult.Complete(Boolean)+19e System_ServiceModel_ni!System.ServiceModel.Channels.SynchronizedMessageSource+ReceiveAsyncResult.OnReceiveComplete(System.Object)+82 System_ServiceModel_ni!System.ServiceModel.Channels.SessionConnectionReader.OnAsyncReadComplete(System.Object)+170 System_ServiceModel_ni!System.ServiceModel.Channels.SocketConnection.OnReceiveAsync(System.Object, System.Net.Sockets.SocketAsyncEventArgs)+fd System_ni!System.Net.Sockets.SocketAsyncEventArgs.FinishOperationSuccess(System.Net.Sockets.SocketError, Int32, System.Net.Sockets.SocketFlags)+bd5 System_ni!System.Net.Sockets.SocketAsyncEventArgs.CompletionPortCallback(UInt32, UInt32, System.Threading.NativeOverlapped*)+40 mscorlib_ni!System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32, UInt32, System.Threading.NativeOverlapped*)+96 [[GCFrame]] [[DebuggerU2MCatchHandlerFrame]] [[ContextTransitionFrame]] [[DebuggerU2MCatchHandlerFrame]]

Could too much data also cause this? Thank you,

R Davies
  • 1
  • 2

0 Answers0