5
var res = new int[1000000].Skip(999999).First();

It would be great if this query would just use the indexer instead of traversing 999999 entries.

I had a look into the System.Core.dll and noticed that in contrast to Skip(), the Count() extension method is optimized. If the IEnumerable implements ICollection then it just calls the Count property.

Michael Petrotta
  • 59,888
  • 27
  • 145
  • 179
codymanix
  • 28,510
  • 21
  • 92
  • 151

2 Answers2

3

I'll let Jon Skeet answer this:

If our sequence is a list, we can just skip straight to the right part of it and yield the items one at a time. That sounds great, but what if the list changes (or is even truncated!) while we're iterating over it? An implementation working with the simple iterator would usually throw an exception, as the change would invalidate the iterator. This is definitely a behavioural change. When I first wrote about Skip, I included this as a "possible" optimization - and actually turned it on in the Edulinq source code. I now believe it to be a mistake, and have removed it completely.

...

The problem with both of these "optimizations" is arguably that they're applying list-based optimizations within an iterator block used for deferred execution. Optimizing for lists either upfront at the point of the initial method call or within an immediate execution operator (Count, ToList etc) is fine, because we assume the sequence won't change during the course of the method's execution. We can't make that assumption with an iterator block, because the flow of the code is very different: our code is visited repeatedly based on the caller's use of MoveNext().

https://msmvps.com/blogs/jon_skeet/archive/2011/01/26/reimplementing-linq-to-objects-part-40-optimization.aspx

Community
  • 1
  • 1
IAmTimCorey
  • 16,412
  • 5
  • 39
  • 75
  • 1
    Your quote is about lists; OP's example is about arrays. Since arrays are fixed-length, they cannot be changed during iteration and their enumerator throws no exceptions. – Gabe Jun 05 '11 at 19:13
  • @Gabe - Actually, the title of that article is "What can we not optimize in LINQ to Objects?" so I guess I would disagree with you since this article is about using Skip() in Linq to Objects – IAmTimCorey Jun 05 '11 at 19:14
  • I always thought, that LINQ IEnumerables should be considered read only. I still do not get it. The optimized version of Skip would still returning a Enumerator (Just starting at index n), thus behaving correctly if the List changes.. – codymanix Jun 05 '11 at 19:17
  • Still, you've answered why lists aren't optimized, but that answer doesn't hold for arrays. Since the OP's example uses an array, you haven't answered the question very well. – Gabe Jun 05 '11 at 19:19
  • @codymanix: A proper implementation of `Skip` on a `List` should throw an exception if somebody modifies the `List` while you're iterating over it. A naive optimization of `Skip` won't do that, as Jon Skeet explains. Since an array (as in your example) doesn't throw exceptions during iteration, there's no reason that can't be optimized. – Gabe Jun 05 '11 at 19:21
  • Now I understand. We cannot use the internal version of List without using the standard Enumerator from List. Too bad.. – codymanix Jun 05 '11 at 19:32
  • @codymanix: If you look at my answer, you'll see that it's easy to write your own Skip function that does the optimization properly. – Gabe Jun 05 '11 at 19:44
3

If you look at my answer to a similar question, it appears as though it should be easy to provide a non-naive (i.e. throws proper exceptions) optimization of Skip for any IList:

    public static IEnumerable<T> Skip<T>(this IList<T> source, int count)
    {
        using (var e = source.GetEnumerator())
            while (count < source.Count && e.MoveNext())
                yield return source[count++];
    }

Of course, your example uses an array. Since arrays don't throw exceptions during iteration, even doing anything as complicated as my function would be unnecessary. One could thus conclude that MS didn't optimize it because they didn't think of it or they didn't think it was a common enough case to be worth optimizing.

Community
  • 1
  • 1
Gabe
  • 84,912
  • 12
  • 139
  • 238
  • I also though about misusing MoveNext() of the Lists iterator but it seem more like a hack. Great idea anyway. – codymanix Jun 05 '11 at 20:08