7

From my ASP.NET Core 2.2 application, I am having to call a 3rd party library that is prone to pushing the CPU to 100% and basically hanging the machine - happens at least twice a month. I have no access to the source nor will the vendor fix it.

My solution to this problem was to isolate this 3rd party library in a .NET Framework 4.x web service where I can call Thread.Abort it if I detect issues. The reason for isolating it in a .NET Framework service rather than .NET Core, is because the latter doesn't support Thread.Abort. The current solution, while not ideal, works. Even knowing that Thread.Abort could cause instability (hasn't so far).

I'd rather not have the library isolated for performance reasons. But so far, I haven't found a way to kill a runaway thread (or Task) in a .NET Core project.

What are the alternatives that are available to me?

Stephen Kennedy
  • 20,585
  • 22
  • 95
  • 108
AngryHacker
  • 59,598
  • 102
  • 325
  • 594
  • have you look for cancellation https://learn.microsoft.com/en-us/dotnet/standard/parallel-programming/task-cancellation – Maytham Fahmi Oct 01 '19 at 20:25
  • My thought as well, were the cancelation tokens tried? https://learn.microsoft.com/en-us/dotnet/standard/threading/cancellation-in-managed-threads – Austin T French Oct 01 '19 at 20:36
  • 3
    @maytham-ɯɐɥʇʎɐɯ Cancellation tokens only work if u have access to the code that causes the problem - e.g something to listen for cancellation. The issues hanging the machine are entirely in the 3rd party library. – AngryHacker Oct 01 '19 at 20:41
  • 2
    @AngryHacker Now I become curious like you to find the solution for such issue. hope some one can enrich us with some useful info. – Maytham Fahmi Oct 01 '19 at 20:47
  • 5
    Isolate it in its own process, then kill the process. – Lasse V. Karlsen Oct 01 '19 at 20:52
  • There is no "valid" solution for third-party libraries I guess in this case – misticos Oct 01 '19 at 20:54
  • 2
    @LasseVågsætherKarlsen That's effectively what I am doing now - not a huge fan of this approach. – AngryHacker Oct 01 '19 at 20:55
  • Have you considered [restarting the ASP.Net application programmatically](https://stackoverflow.com/questions/36121999/how-to-restart-asp-net-core-app-programmatically) when issues are detected? – Theodor Zoulias Oct 01 '19 at 21:11
  • @TheodorZoulias I'd rather not restart the entire app, but this might be an idea to pursue. – AngryHacker Oct 01 '19 at 21:34
  • Did you tried to decompile 3rd party library? Maybe that allow you to fix the root cause – Kalten Oct 01 '19 at 22:09

2 Answers2

4

I also agree with the comment that tearing down the whole process might be a more clean solution in this case. However, if you prefer to stick with the Thread.Abort approach, it's not difficult to implement it with .NET Core for Windows at least, using Win32 Interop to call unmanaged TerminateThread API.

Below is an example of doing that (warning: almost untested).

using System;
using System.Runtime.InteropServices;
using System.Threading;
using System.Threading.Tasks;

namespace CoreConsole
{
    class Program
    {
        static async Task Main(string[] args)
        {
            try
            {
                using (var longRunningThread = new LongRunningThread(() => Thread.Sleep(5000)))
                {
                    await Task.Delay(2500);
                    longRunningThread.Abort();
                    await longRunningThread.Completion;
                    Console.WriteLine("Finished");
                }
            }
            catch (Exception ex)
            {
                Console.WriteLine($"{ex.Message}");
            }
        }
    }

    public class LongRunningThread : IDisposable
    {
        readonly Thread _thread;

        IntPtr _threadHandle = IntPtr.Zero;

        readonly TaskCompletionSource<bool> _threadEndTcs;

        readonly Task _completionTask;

        public Task Completion { get { return _completionTask; } }

        readonly object _lock = new object();

        public LongRunningThread(Action action)
        {
            _threadEndTcs = new TaskCompletionSource<bool>(TaskCreationOptions.RunContinuationsAsynchronously);

            _thread = new Thread(_ =>
            {
                try
                {
                    var hCurThread = NativeMethods.GetCurrentThread();
                    var hCurProcess = NativeMethods.GetCurrentProcess();
                    if (!NativeMethods.DuplicateHandle(
                        hCurProcess, hCurThread, hCurProcess, out _threadHandle,
                        0, false, NativeMethods.DUPLICATE_SAME_ACCESS))
                    {
                        throw new System.ComponentModel.Win32Exception(Marshal.GetLastWin32Error());
                    }

                    action();

                    _threadEndTcs.TrySetResult(true);
                }
                catch (Exception ex)
                {
                    _threadEndTcs.TrySetException(ex);
                }
            });

            async Task waitForThreadEndAsync()
            {
                try
                {
                    await _threadEndTcs.Task.ConfigureAwait(false);
                }
                finally
                {
                    // we use TaskCreationOptions.RunContinuationsAsynchronously for _threadEndTcs
                    // to mitigate possible deadlocks here
                    _thread.Join();
                }
            }

            _thread.IsBackground = true;
            _thread.Start();

            _completionTask = waitForThreadEndAsync();
        }

        public void Abort()
        {
            if (Thread.CurrentThread == _thread)
                throw new InvalidOperationException();

            lock (_lock)
            {
                if (!_threadEndTcs.Task.IsCompleted)
                {
                    _threadEndTcs.TrySetException(new ThreadTerminatedException());
                    if (NativeMethods.TerminateThread(_threadHandle, uint.MaxValue))
                    {
                        NativeMethods.WaitForSingleObject(_threadHandle, NativeMethods.INFINITE);
                    }
                    else
                    {
                        throw new System.ComponentModel.Win32Exception(Marshal.GetLastWin32Error());
                    }
                }
            }
        }

        public void Dispose()
        {
            if (Thread.CurrentThread == _thread)
                throw new InvalidOperationException();

            lock (_lock)
            {
                try
                {
                    if (_thread.IsAlive)
                    {
                        Abort();
                        _thread.Join();
                    }
                }
                finally
                {
                    GC.SuppressFinalize(this);
                    Cleanup();
                }
            }
        }

        ~LongRunningThread()
        {
            Cleanup();
        }

        void Cleanup()
        {
            if (_threadHandle != IntPtr.Zero)
            {
                NativeMethods.CloseHandle(_threadHandle);
                _threadHandle = IntPtr.Zero;
            }
        }
    }

    public class ThreadTerminatedException : SystemException
    {
        public ThreadTerminatedException() : base(nameof(ThreadTerminatedException)) { }
    }

    internal static class NativeMethods
    {
        public const uint DUPLICATE_SAME_ACCESS = 2;
        public const uint INFINITE = uint.MaxValue;

        [DllImport("kernel32.dll")]
        public static extern IntPtr GetCurrentThread();

        [DllImport("kernel32.dll")]
        public static extern IntPtr GetCurrentProcess();

        [DllImport("kernel32.dll", SetLastError = true)]
        public static extern bool CloseHandle(IntPtr handle);

        [DllImport("kernel32.dll", SetLastError = true)]
        public static extern bool DuplicateHandle(IntPtr hSourceProcessHandle,
           IntPtr hSourceHandle, IntPtr hTargetProcessHandle, out IntPtr lpTargetHandle,
           uint dwDesiredAccess, bool bInheritHandle, uint dwOptions);

        [DllImport("kernel32.dll", SetLastError = true)]
        public static extern bool TerminateThread(IntPtr hThread, uint dwExitCode);

        [DllImport("kernel32.dll", SetLastError = true)]
        public static extern uint WaitForSingleObject(IntPtr hHandle, uint dwMilliseconds);
    }

}
noseratio
  • 59,932
  • 34
  • 208
  • 486
  • @AngryHacker, glad if it helped, although it will probably leak memory/resources if used on the regular basis. Why are you not happy with recycling it as a process (vs thread)? – noseratio Oct 02 '19 at 22:14
  • 1
    This problem I am running into is a minor part of the application functionality. I'd rather not disrupt the entire app by restarting the entire process for this minor portion. – AngryHacker Oct 02 '19 at 23:03
  • 1
    @AngryHacker, also see if you can use `ExitThread` instead of `TerminateThread`. Depending on what that naughty library is doing inside its tight loop, you might be able to inject a `ExitThread` call to be executed on the target thread via something like `QueueUserAPC` or `WH_GETMESSAGE` hook. – noseratio Oct 02 '19 at 23:29
2

You could lower the Thread.Priority, which is available in Core 3.0. It'll still use all available CPU cycles when nothing else needs them but the system will be more responsive.

Ed Power
  • 8,310
  • 3
  • 36
  • 42