21

In the following test:

int[] data = { 1, 2, 3, 4, 5, 6, 7, 8, 9, 10 };
Func<int, int> boom = x => { Console.WriteLine(x); return x; };
var res = data.Select(boom).Skip(3).Take(4).ToList();
Console.WriteLine();
res.Select(boom).ToList();

The result is:

1
2
3
4
5
6
7

4
5
6
7

Essentially, I observed that in this example, Skip() and Take() work well, Skip() is not as lazy as Take(). It seems that Skip() still enumerates the items skipped, even though it does not return them.

The same applies if I do Take() first. My best guess is that it needs to enumerate at least the first skip or take, in order to see where to go with the next one.

Why does it do this?

casperOne
  • 73,706
  • 19
  • 184
  • 253
ericosg
  • 4,926
  • 4
  • 37
  • 59
  • 4
    Well, `Skip(n)` works by asking for the next element, `n` times. The next element comes from your select. Some collections might make sense to 'jump ahead', but not all. What if the collection was generated by a custom enumerator? – Rob Dec 03 '15 at 05:08
  • 4
    `Skip` is not increasing index by `n` or something like this. `Skip` is just telling enumerator "give a next item" `n` times without storing these values in a result array. – Yeldar Kurmangaliyev Dec 03 '15 at 05:11
  • 1
    @YeldarKurmangaliyev, i think your explanation makes the most sense. @Rob, indeed, a custom enumerator that actually never inspects the lazy item would be better here. I think that's how Skip/Take should work. But the iterator in `Where()` is even worse. – ericosg Dec 03 '15 at 05:14
  • 2
    @ericosg I wasn't advocating a custom enumerator for skip, I was saying that it's *not possible* for skip to directly jump `n` items ahead, because of things like custom enumerators. – Rob Dec 03 '15 at 05:20
  • 1
    @ericosg As for `Where`, it should be clearer why it needs to iterate each item. What if it were `Where(a => a == 1)` and `boom` was yielding random numbers between `1-5`? `Where` would *have* to iterate each item to get all the ones with the value of `1`. – Rob Dec 03 '15 at 05:23
  • 1
    @Rob, yeah that's where a `Where()` overload with only an iterator index would be handy. – ericosg Dec 03 '15 at 05:26
  • 1
    @ericosg Sorry, what do you mean exactly? – Rob Dec 03 '15 at 05:29
  • 1
    @Rob, if there was a `Where` which had a 3rd overload, (i.e. `Where(Func predicate)` where `int` is the element source index, it would be useful in my case. – ericosg Dec 03 '15 at 05:33
  • 2
    @ericosg You can write it yourself as an extension :) - Should essentially just be `public static IEnumerable Where(this IEnumerable source>, Func predicate, int index) { return source.Skip(index).Where(predicate); }` - and then use it just like any other LINQ method – Rob Dec 03 '15 at 05:36
  • 15
    **Do not make projections that have side effects**. Projections exist to *produce a result*, not a side effect. The behaviour of a projection with a side effect can be very bizarre indeed, as you have discovered. – Eric Lippert Dec 03 '15 at 05:51

6 Answers6

22

Skip() and Take() both operate on IEnumerable<>.

IEnumerable<> does not support skipping ahead -- it can only give you one item at a time. With this in mind, you can think of the Skip() more as a filter -- it still touches all the items in the source sequence, but it filters out however many you tell it to. And importantly, it filters them out from getting to whatever is next, not for whatever is in front of it.

So, by doing this:

data.Select(boom).Skip(3)

You are performing boom() on each item before they get to the Skip() filter.

If you instead changed it to this, it would filter prior to the Select and you would call boom() on only the remaining items:

data.Skip(3).Take(4).Select(boom)
Cory Nelson
  • 29,236
  • 5
  • 72
  • 110
  • I guess it's still better than `Where`, since that iterates through all 10 to get the iterator. i.e. `var res = data.Select(boom).Where((x,i) => i >= 3 && i < 7).ToList();` vs `var res = data.Where((x,i) => i >= 3 && i < 7).Select(boom).ToList();` – ericosg Dec 03 '15 at 05:12
  • 1
    There should be virtually no difference between `Where()` and `Skip()` in terms of items touched. There will be a marked difference between `Where()` and `Take()`, though.. – Cory Nelson Dec 03 '15 at 05:15
  • indeed that's the case. – ericosg Dec 03 '15 at 05:16
5

If you decompile Enumerable, you will see the following implementation of Skip:

while (count > 0 && e.MoveNext())
  --count;

and the following implementation of Take:

foreach (TSource source1 in source)
{
  yield return source1;
  if (--count == 0) break;
}

So, both of these LINQ methods actually enumerate through these items. The difference is whether an enumerated item will be placed in the resulting collection or not. That's how IEnumerable does work.

Yeldar Kurmangaliyev
  • 33,467
  • 12
  • 59
  • 101
2

They both iterate the collection. And then turnout with the final collection. Its just like a simple for loop having a if else condition in it.

Moreover you're selecting it first using boom, it prints the item of collection.

By this example you can not tell if skip() or take() iterates the whole collection or not but the matter of fact, they do.

Shaharyar
  • 12,254
  • 4
  • 46
  • 66
1

assuming i have understood your question correctly.

the boom func executes before the skip operation.

try to select data after skip and take , you will get the exact.

karthikkumar subramaniam
  • 358
  • 1
  • 3
  • 19
  • Yes indeed, I know that if I Skip/Take after my select it works as expected, but my ponder is as to why the visa-versa doesn't. – ericosg Dec 03 '15 at 05:10
  • coming for your question, while performing the result for var res = data.Select(boom).Skip(3).Take(4).ToList(); the boom function executed seven times totally ( 3 for skip and 4 for take) so it prints. and the final result from the filtered array. – karthikkumar subramaniam Dec 03 '15 at 05:40
1

This behaviour maybe will be changed in the future, as can be seen here in this discussion about a Pull Request that optimizes the behaviour of the Skip method for sources that implement the IList interface.

T_D
  • 1,688
  • 1
  • 17
  • 28
0

Not an answer, but think about how Skip(...) would be implemented. Here's one way to do it:

public static IEnumerable<T> Skip<T>(this IEnumerable<T> list, int count)
{
    foreach (var item in list)
    {
        if (count > 0) count--;
        else yield return item;
    }
}

Notice that the whole list argument is enumerated even though only a subset is returned.

user2023861
  • 8,030
  • 9
  • 57
  • 86