IIS Application Pool Crashing - Safe handle has been closed - IAsync

Question

We have a production issue that is crashing the IIS application pool for a .NET 4.8 application. The EventViewer shows the application pool crashing and a corresponding error:

An unhandled exception occurred and the process was terminated.

Application ID: /LM/W3SVC/3/ROOT

Process ID: 18864 Exception: System.ObjectDisposedException Message: Safe handle has been closed

StackTrace: at System.Runtime.InteropServices.SafeHandle.DangerousAddRef(Boolean& success) at System.StubHelpers.StubHelpers.SafeHandleAddRef(SafeHandle pHandle, Boolean& success) at Microsoft.Win32.Win32Native.SetEvent(SafeWaitHandle handle) at System.Threading.EventWaitHandle.Set() at System.Runtime.Remoting.Messaging.AsyncResult.SyncProcessMessage(IMessage msg) at System.Runtime.Remoting.Messaging.StackBuilderSink.AsyncProcessMessage(IMessage msg, IMessageSink replySink) at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx) at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem() at System.Threading.ThreadPoolWorkQueue.Dispatch()

I believe it is related to the use of IAsyncResult given the AsyncProcessMessage and the application has only 1 usage but nothing jumps out. I've used DebugDiag which didn't reveal any additional details.

var swTimer = Stopwatch.StartNew();
AsyncMethodCaller caller = new AsyncMethodCaller(myProg.DoWork);
IAsyncResult delegateResult = caller.BeginInvoke();

while (delegateResult.IsCompleted == false)
{
    Thread.Sleep(100);
    if (swTimer.ElapsedMilliseconds > 50000)
    {
        logger.LogTrace("Long Running Process");
        // Update DB with status
        swTimer.Restart();
    }
}

result = caller.EndInvoke(delegateResult);
delegateResult.AsyncWaitHandle.Close();

score 0 · Answer 1 · answered Mar 24 '22 at 03:36

I try to reproduce the issue according to the code you provide but failed. So I'm not sure if the issue was caused by above code. How did you know for sure?

However, I find something strange in the code.

IAsyncResult delegateResult = caller.BeginInvoke();

When you use BeginInvoke() to call the delegate mthod, you need to pass two paramters at least AsyncCallback and object, even though the synchronous method does not need to pass in any parameters.

delegateResult.AsyncWaitHandle.Close();

Why you call WaitHandle.Close to release resource while there's no call of AsyncWaitHandle property? Only when you use

 // Wait for the WaitHandle to become signaled.
            result.AsyncWaitHandle.WaitOne();

to wait a singal from caller, you need to close it for release resource.

The wait handle is not closed automatically when you call EndInvoke. If you release all references to the wait handle, system resources are freed when garbage collection reclaims the wait handle. To free the system resources as soon as you are finished using the wait handle, dispose of it by calling the WaitHandle.Close method. Garbage collection works more efficiently when disposable objects are explicitly disposed.

Waiting for an Asynchronous Call with WaitHandle

// Update DB with status

I'm not sure what code it is. But if you use thread of caller to update DB, this may causes thread blocking and your issue.

Thank you for the help. I remove the BeginInvoke's parameter just to simply the logic. The Update DB with status was code removed that just updates the SQL DB via dapper. This logic is necessary to update the DB with the processing status: var step = _masterRepository.GetStep(myId, "EXT").Result; await _masterRepository.UpdateStep(step).ConfigureAwait(false); It's important to note that this issue appears to be random and only seems to occur when called multiple times in a row. Also, I am not entirely sure this code is suspect given the stack trace doesn't provide much detail. — sagosto, Mar 25 '22 at 04:25
Can you describe more about "this issue appears to be random"? I mean does it appear when access some specific pages or access all page with multiple times? — Bruce Zhang, Mar 25 '22 at 05:21
The issue is within a .NET 4.8 REST API layer when multiple calls occur to perform a long running process that essentially pulls data out of the database into XML/PDF files. It ONLY seems to occur when multiple concurrent processing is being performed and there does not appear to be any consistent way to reproduce. I was able to reproduce it locally last night and VS2022 just crashed. Nothing in the Nlogs but EventViewer picked it up -- Original post has the stack trace. — sagosto, Mar 25 '22 at 13:38
It seems that the async code is causing the thread to clutter. Have you tried using synchronous code? — Bruce Zhang, Apr 06 '22 at 07:08

IIS Application Pool Crashing - Safe handle has been closed - IAsync

1 Answers1