Do something for every hundredth element in an array

Question

I have 1 million elements in an array, I need to divide these elements into groups of 100, do a function and continue working on the next hundred

foreach (string value in lines)
{
    filescreated++;
    if (filescreated == ?????????)
    {
        do stuff 
    }
}

???? is equal to value divisible by 100

Do you need to have every group stored somewhere so they could be accessed later? — as-cii, Sep 15 '11 at 23:30
@tke Give the guy a break, if he knew he wouldn't have asked. The fact that there is *some* code tells me he tried, but couldn't come up with the test condition. And that's good enough for me. Remember that beginner questions *are* allowed on SO. — NullUserException, Sep 15 '11 at 23:31
I'm racking my brains trying to think of how you can do this using Linq. Can anyone think of an answer? — Andrew Shepherd, Sep 15 '11 at 23:39
OK, I've found an answer here: http://stackoverflow.com/questions/3514740/how-to-split-an-array-into-a-group-of-n-elements-each — Andrew Shepherd, Sep 15 '11 at 23:45
@Andrew: That'll perform horribly for a million elements, though... See the notes on their solution. — Reed Copsey, Sep 15 '11 at 23:49
@Reed Copsey - OK, but what if we don't use eager evaluation? I've posted this an answer which uses LINQ but lazy evaluation. — Andrew Shepherd, Sep 15 '11 at 23:51
You need eager evaluation on the inner part, at least: Consider if it were `IEnumerable>` and if you were to just call .ToList() on that, without looking at the inner sequences yet. By necessity the inner part would need to be evaluated to segment the outer part, but now how do you go back and look at those inner sequences once they've been skipped unless they've been buffered? — Mark Sowul, Sep 16 '11 at 01:52

NullUserException · Answer 1 · 2011-09-15T23:52:34.110

is equal to value divisable by 100

foreach (...)
{
    filescreated++;

    if (filescreated % 100 == 0) 
    {
        // do stuff for the every 100th element
    }

    // do other stuff for every element
}

Reference: modulus (%) operator

Use this if you need to do something special for every 100th element, but you still need to process every element.

If you only need to process every 100th element, refer to Reed's answer.

Mathieu · Answer 2 · 2011-09-15T23:43:24.073

What about this (if you need in order and % isn't good for you)?

The question is confusing as you talk about every hundredth element, then after about packs of 100. So here would be a guess.

        string[] lines = new string[1000000];
        for (int i = 0; i < 10000; i++)
        {
            for (int j = 0; j < 100; j++)
            {
                DoSomething(lines[100*i + j], i);
            }
        }

score 0 · Answer 3 · answered Sep 15 '11 at 23:48

0

I need to divide these elements into groups of 100, do a function and continue working on the next hundred

You can do this directly, if this is an array, just by using a for loop and incrementing by 100:

int chunkSize = 100;

for (int start=0; start<lines.Length;start += chunkSize)
{        
    ProcessSectionOfArray(lines, start, Math.Min(start+chunkSize-1, lines.Length-1));
}

answered Sep 15 '11 at 23:48

Reed Copsey

554,122
78
1,158
1,373

I think the OP wants to work on every element, but do something special for every 100th element. – NullUserException Sep 15 '11 at 23:51
@NullUserException: It was unclear - I posted this just in case, as it's much better than looping through the other 99 elements unnecessarily... – Reed Copsey Sep 16 '11 at 00:05

Andrew Shepherd · Answer 4 · 2011-09-16T00:56:15.860

Here is a solution which separates the partitioning logic into a separate function.

    // A separate static function
    private static IEnumerable<IEnumerable<T>> BreakIntoBlocks<T>(T[] source, int blockSize)
    {
        for (int i = 0; i < source.Length; i += blockSize)
        {
            yield return source.Skip(i).Take(blockSize);
        }
    }


    // And in your code

    string[] lines = new string[1000000];
    foreach(IEnumerable<string> stringBlock in BreakIntoBlocks(lines, 100))
    {
            // stringblock is a block of 100 elements
            // Here is where you put the code that processes each separate group
    }

The attempt above should be faster than my first attempt (below)

        int blockSize = 100;
        int i = 0;
        IEnumerable<IEnumerable<string>> query = from s in lines
                                                let num = i++
                                                group s by num / blockSize into g
                                                select g;
        foreach(IEnumerable<string> stringBlock in query)
        {
            // Stringblock will be a block of 100 elements.
            // Process this 100 elements here.
        }

The problem which using the grouping clause is that LINQ will allocate every one of those 1000000 element to groups before it returns the first element.

Do something for every hundredth element in an array

4 Answers4