1

I am studying the internal mechanics of the iterator methods, and I noticed a strange difference in behavior between the IEnumerator<T> obtained by an iterator and the IEnumerator<T> obtained by a LINQ method. If an exception happens during the enumeration, then:

  1. The LINQ enumerator remains active. It skips an item but continues producing more.
  2. The iterator enumerator becomes finished. It does not produce any more items.

Example. An IEnumerator<int> is enumerated stubbornly until it completes:

private static void StubbornEnumeration(IEnumerator<int> enumerator)
{
    using (enumerator)
    {
        while (true)
        {
            try
            {
                while (enumerator.MoveNext())
                {
                    Console.WriteLine(enumerator.Current);
                }
                Console.WriteLine("Finished");
                return;
            }
            catch (Exception ex)
            {
                Console.WriteLine($"Exception: {ex.Message}");
            }
        }
    }
}

Let's try enumerating a LINQ enumerator that throws on every 3rd item:

var linqEnumerable = Enumerable.Range(1, 10).Select(i =>
{
    if (i % 3 == 0) throw new Exception("Oops!");
    return i;
});
StubbornEnumeration(linqEnumerable.GetEnumerator());

Output:

1
2
Exception: Oops!
4
5
Exception: Oops!
7
8
Exception: Oops!
10
Finished

Now let's try the same with an iterator that throws on every 3rd item:

StubbornEnumeration(MyIterator().GetEnumerator());

static IEnumerable<int> MyIterator()
{
    for (int i = 1; i <= 10; i++)
    {
        if (i % 3 == 0) throw new Exception("Oops!");
        yield return i;
    }
}

Output:

1
2
Exception: Oops!
Finished

My question is: what is the reason for this inconsistency? And which behavior is more useful for practical applications?

Note: This observation was made following an answer by Dennis1679 in another iterator-related question.


Update: I made some more observations. Not all LINQ methods behave the same. For example the Take method is implemented internally as a TakeIterator on .NET Framework, so it behaves like an iterator (on exception completes immediately). But on .NET Core it's probably implemented differently because on exception it keeps going.

Boann
  • 48,794
  • 16
  • 117
  • 146
Theodor Zoulias
  • 34,835
  • 7
  • 69
  • 104
  • 1
    The first code example should be clear. Since you're catching every exception, the outer loop doesn't break. The example with `yield return` may look a little weird, but the compiler does a lot behind the scenes to make it work like that. – Dennis_E Oct 06 '19 at 12:21
  • @Dennis_E the same method `StubbornEnumeration` is used to enumerate both enumerators, the LINQ one and the iterator one. And the results are different. I didn't expect this difference to exist to be honest. – Theodor Zoulias Oct 06 '19 at 12:24

1 Answers1

0

The yield syntax in the second example makes the difference. When you use it, the compiler generates a state machine that manages a real enumerator under the hood. Throwing the exception exits the function and therefore terminates the state machine.

Rex M
  • 142,167
  • 33
  • 283
  • 313
  • Is there a way to rewrite my iterator so that it behaves like the LINQ enumerables? – Theodor Zoulias Oct 06 '19 at 12:20
  • Yes, you can build your own implementation of `IEnumerable` and `IEnumerator` (which is exactly what LINQ has done under the hood) that allows exceptions to bubble out of `MoveNext()` without setting its internal state to finished. – Rex M Oct 06 '19 at 12:52
  • Do you have any explanation why the iterators are implemented in a way that is not consistent with the way an everyday developer would code by hand a `IEnumerable`/`IEnumerator` pair? – Theodor Zoulias Oct 06 '19 at 12:57
  • Both are reasonable based on their context. In the LINQ case, we are throwing the exception from inside the lambda, which explicitly runs once per iteration. In the `yield` case, if we throw an exception but somehow kept running the function instead of leaving it, there would be a lot of questions on here trying to understand and explain that behavior! – Rex M Oct 06 '19 at 13:28
  • I made this experiment. I copy-pasted my iterator to [SharpLab.io](https://sharplab.io/#v2:CYLg1APgAgTAjAWAFBQAwAIpwCwG5nJQDMmM6AsgJ4DCANgIYDOj6A3gUul5nAGw9EAPAEsAdgBcAfBUoBJcQFMATvXEB7JQAoAlMm5s9+7gDMN6TWPHph6ALzo4ua+kH24qJ8LBhdnI13Y/f31hY3MbAFJ0Elt7VG10cQALJTUAd3RRBQyAUQAPAGMFAAdxYTVRTQAiAHk1YsYAQirtfCDgriw4TAB2azaOgF9DLmGkZDGgA===), took the generated code, fixed the illegal variable names to make it compile, and used it as an argument for `StubbornEnumeration`. It behaves like an iterator. – Theodor Zoulias Oct 06 '19 at 13:55
  • Then I commented out the two lines `<>1__state = -1;` inside the generated `MoveNext` method, and started behaving like a LINQ enumerator. So there is explicit code in the compiler-generated class to make it behave in an non-intuitive way. This is puzzling. – Theodor Zoulias Oct 06 '19 at 13:56
  • Like I said in my previous comment, the code is there to make it behave in the most intuitive way, which is if you throw an exception in a method, the method exits and it will not execute any more code. What you are trying to create is a situation where, because you are using `yield`, throwing an exception does something different where the caller receives an exception but the method somehow continues to run. – Rex M Oct 06 '19 at 14:04
  • You have a point. Personally though I've never considered iterators as normal methods that are entered and exited only once per call. I always considered them as a shortcut for creating enumerables. It is much quicker and easier to code an iterator than to code a pair of `IEnumerable`/`IEnumerator` implementations. – Theodor Zoulias Oct 06 '19 at 14:12
  • After giving it some thought, I don't thing that your point is compelling. The situation in which an iterator behaves differently than a hand-coded enumerable in not trivial. An unsuspicious developer will not be surprised, because she will not handle an enumerator explicitly to begin with. She will use `foreach`, which creates an implicit enumerator and disposes it on exception, not allowing the scenario we are talking here to occur. So I wouldn't put my money on this explanation. – Theodor Zoulias Oct 07 '19 at 06:40