4

Here's a puzzle for you guys. I have a multithreaded program in which some threads work with managed resources like locks and semaphores. Some of the lock release primitives can only be performed from the same thread on which the lock acquire was done.

So here's my puzzle: I wrap these kinds of operations: try { lock-acquire... do something } finally { lock-release }, but sometimes when my threads terminate, the finally clauses are performed by the .NET garbage collection thread and not by my thread. (The specific case actually involves the Dispose of an object allocated in a "using" statement; see below for details)

This is a little tricky to demo; I see it all the time in Isis2, and I've figured out that this is happening by checking the thread-ids in the acquire and finalize blocks. But I don't have a 3-line demo for you, and I am sorry about that. I know it would make it easier to provide help.

Is there a way I can delay termination for a thread until all pending finalize blocks associated with that thread have executed?

---- Details added for Mark ----

What I'm really doing involves a fairly elaborate self-instrumenting locking package that has various locking abstractions (bounded buffers, barriers, normal locks, etc) with thread priorities and designed to self-detect deadlocks, priority inversions, very slow lock grants and other problems. My code is complex enough and threaded enough so that I needed this to debug it.

An example of my basic style of construct is this:

LockObject  myLock = new LockObject("AnIsis2Lock");

...

using(new LockAndElevate(myLock)) { stuff }

A LockObject is a self-monitoring lock... a LockAndElevate makes note that I locked it (in this case) and later tracks the dispose, when I unlock it. So I'm exploiting the fact that using should dispose of the new object when the block completes -- even if it throws an exception.

What I'm seeing is that fairly often, threads terminate and yet the using dispose events haven't actually occurred; they run later, on some other thread. This only happens when one of my threads terminates; in normal execution, the whole thing works like a charm.

So since using translates to try... finally, my question was posted in terms of finally clauses.

Ken Birman
  • 1,088
  • 8
  • 25
  • I wasn't aware that finalize blocks are "associated with a thread" – John Saunders Jul 18 '12 at 21:31
  • 5
    Note, `"Finalize" != "finally"`. As worded, the question sounds like you're a bit confused about that, and mixing up the two. If you mean the latter, then that does happen automatically, and on the same thread, unless the app crashes or is in the process of doing so. – cHao Jul 18 '12 at 21:45
  • I'm using "finally" as in "try { } finally { }. And I regret to say that you are incorrect. There are definitely conditions (I'm trying to pin down just when) in which a finally{ } clause executes on some other thread. In fact I actually work with "using" but the two constructs are the same per the C# reference manual. – Ken Birman Jul 18 '12 at 22:16
  • 5
    `I myself never used the word "finalize"` Well... apart from the three times you used it in your question. And once more in the title. And also you tagged the question "finalize". So I'm not really sure what you mean by this statement. – Mark Byers Jul 18 '12 at 23:41
  • 6
    @Ken, `finally` will never run on a separate thread. This is imperative programming -- each statement within a method runs sequentially on the same thread. (`Finalize` on the other hand, would be a completely different story) – Kirk Woll Jul 19 '12 at 01:00
  • Kirk, I assumed this to be true and it certainly is true when my threads don't terminate. But there seems to be some form of thrown exception handling in which the finally clauses get executed later -- more precisely, the Dispose actions for my objects get executed later, and on a different thread. The deal is that my threads are trying to shut down gracefully (I'm seeing this when my library shuts down, not in normal mode), threads throw "IsisShutdown" exceptions, which they catch at the top level, then gracefully do a return to exit the top-level context. This (perhaps) causes my issue. – Ken Birman Jul 19 '12 at 01:03
  • Mark, ok, I used it in a vernacular sense, but not in the sense of the language construct, as was clear from my paragraph! But I edited to fix that because it confused people. My bad. – Ken Birman Jul 19 '12 at 01:07
  • Do your objects have finalizers (e.g. `~LockObject()`)? How exactly are the threads terminated? Aren't your methods using `yield return`? – svick Jul 19 '12 at 01:11
  • The .NET documentation shows a little incantation you use to define a Dispose method and arrange that no matter how the object gets deleted, it will be called; I based my solution on that code. As for the threads: at the very top, most of them have a try { do-stuff } catch (IsisShutdown) { } and my guess is that the bad behavior occurs precisely when an IsisShutdown is thrown. They then just do a return, which will terminate the thread. Would a yield return be preferable? At any rate, the Dispose methods haven't executed for some objects after the thread is long gone. – Ken Birman Jul 19 '12 at 01:16
  • Well, meanwhile, my working hypothesis (that caught exceptions can sometimes prevent the execution of Dispose methods on C# 5 + .NET 4) seems to have led me to a way to handle this problem, as a nasty sort of hack. But it seems to me that this is very much NOT the behavior documented for the "using" statement... – Ken Birman Jul 19 '12 at 14:08
  • @KirkWoll There are at least two cases where `finally` can run on a different thread: `yield` iterators and `async`/`await`. But I think you're right that the OP mixes up `Finalize()` and `finally.` – CodesInChaos Jul 21 '12 at 12:07
  • CiC: I'm not mixing them up. In the real use case, with a "using(x = new something()) { }, using is mapped by the c# compiler to a try/finally structure that disposes x. But there are also situations (see my answer below) where C# instead calls the finalizer, and it in turn calls the dispose method. I had hoped to not have to explain all this detail but clearly, doing so helps people understand the issue better. So my mistake, but not my confusion. – Ken Birman Jul 21 '12 at 12:51
  • @CodesInChaos, I thought about those two examples, but I couldn't think of any way in which they would run on a separate thread. 1) `yield` iterators are still sequential, the sequence is just unorthodox. How would a separate thread get involved? 2) Does `await` really not return you to the original thread when the async stuff is done? i.e. does `await` break `[ThreadStatic]`, `HttpContext.Current`, etc? (I confess I haven't had the chance to practice that feature yet.) – Kirk Woll Jul 21 '12 at 16:57
  • @KirkWoll 1) If you call `MoveNext` on different threads(which you're allowed to do, if the calls don't overlap) the code gets executed on different threads 2) By default it does return to syncronization context. If that corresponds to the original thread depends on the sync context. On the UI thread of WinForms applications, it'll return to the original thread. For thread pool threads it wont. You can use `ConfigureAwait` to tell it not to return to the original SynchronizationContext. – CodesInChaos Jul 21 '12 at 16:59
  • @CodesInChaos, cool, thanks for elaborating. – Kirk Woll Jul 21 '12 at 17:00

1 Answers1

2

So as of now, here's my best answer to my own question, based mostly on experience debugging the behavior of Isis2.

If threads don't terminate, "using(x = new something()) { }" (which maps to "try { x= new something(); ...} finally { x.dispose }") works precisely as you would expect: the dispose occurs when the code block is exited.

But exceptions disrupt the control flow. So if your thread throws an IsisException, or something down in "something" does so, control passes to the catch for that exception. This the case I'm dealing with, and my catch handler is higher on the stack. C#/.NET faces a choice: does it catch the IsisException first, or does it do the dispose first? In this case I'm fairly certain the system systematically does the IsisException first. As a result, the finalizer for the allocated object "x" has not yet executed but is runable and needs to be called soon.

[NB: For those who are curious, the Using statement ends by calling Dispose, but the recommended behavior, per the documentation, is to have a finalizer ~something() { this.Dispose; } just to cover all possible code paths. Dispose might then be called more than once and you should keep a flag, locked against concurrency, and dispose your managed resources only on the first call to Dispose.]

Now the key problem is that the finalizer apparently might not run before your thread has a chance to terminate in this case of a caught exception that terminates your thread; if not, C# will dispose of the object by calling dispose on a GC finalizer thread. As a result if, as in my code, x.Dispose() unlocks something an error can occur: a lock acquire/release must occur in the same thread on .NET. So that's a source of potential bugs for you and for me! It seems that calling GC.WaitForFinalizers in my exception handler helps in this case. Less clear to me is whether this is a true guarantee that bad things won't occur.

A further serious mistake in my own code was that I had erroneously been catching ThreadAbortException in a few threads, due to an old misunderstanding about how those work. Don't do this. I can see now that it causes serious problems for .NET. Just don't use Thread.Abort, ever.

So on the basis of this understanding, I've modified Isis2 and it works nicely now; when my threads terminate the finalizers do appear to run correctly, dispose does seem to happen before the thread exits (and hence before its id can be reused, which was causing me confusion), and all is good for the world. Those who work with threads, priorities, and self-managed locks/barriers/bounded buffers and semaphores should be cautious: there are dragons lurking here!

Ken Birman
  • 1,088
  • 8
  • 25