Splitting a list or collection into chunks

Question

I have a program that will read the contents of a file into some sort of list or array. This list/array could contain any number of items. I need to split it into smaller groups, say of 50 items each and then do some processing on each item in each group.

List<string> stuffFromFile = new List<string>();

while ((line = fileReader.ReadLine()) != null)
      {
           stuffFromFile.Add(line);
      }

I have been looking through some examples online about how to chunk stuff but to be honest, I don't really understand the examples and some of them seem overly complex. I just need something simple that will chunk/split/break the original list of items into groups of 50 and then let me iterate through each item in each group until the processing is complete.

The total number of items read in will most likely not be a number that I can divide evenly by 50 so most likely the last group may contain less than 50 items, but would still need to be processed just like the rest.

Can anyone help here? It sounds like it should be simple but I don't really know how to do it. I've seen example about using LINQ but I don't understand it either.

I think this might be what you are looking for: http://stackoverflow.com/questions/11463734/split-a-list-into-smaller-lists-of-n-size — Malte R, May 14 '15 at 21:50
why do you want to split into groups when the end goal is to process each item? — imlokesh, May 14 '15 at 21:54
The system processing the data can only accept stuff in small groups. — irldev, May 14 '15 at 21:57

score 28 · Accepted Answer · answered May 14 '15 at 21:53

28

Here's an extension method that will work with any list and any size chunks.

public static List<List<T>> SplitList<T>(this List<T> me, int size = 50)
{
    var list = new List<List<T>>();
    for (int i = 0; i < me.Count; i += size)
        list.Add(me.GetRange(i, Math.Min(size, me.Count - i)));
    return list;
}

Use it like this:

List<List<string>> chunksOf50 = stuffFromFile.SplitList();

answered May 14 '15 at 21:53

Zer0

7,191
1
20
34

How do I iterate through each item in that chunksOf50 list? – irldev May 14 '15 at 22:00
@irldev `foreach (List chunk in chunksOf50)` – Zer0 May 14 '15 at 22:02
for you, something simple like nested foreach loops. `foreach(List listOfStrings in chunksOf50){ foreach(string chunk in listOfStrings){//do stuff}};` – Lee Harrison May 14 '15 at 22:04

score 4 · Answer 2 · answered May 14 '15 at 22:16

List<string> stuffFromFile = new List<string>() { "1", "2", "3", "4" }; //contents

while (stuffFromFile.Count > 0)
{
    List<string> newChunk = stuffFromFile.Take(50).ToList(); //Take up to 50 elements
    stuffFromFile.RemoveRange(0, newChunk.Count); // Remove the elements you took
}

Splitting a list or collection into chunks

2 Answers2