0

So I have a list of array of strings like below:

        List<string[]> ArrayList = new List<string[]>();
        ArrayList.Add(new string[] { "7112432","Gagan","Human Resource"});
        ArrayList.Add(new string[] { "7112433", "Mukesh", "Information Technology" });
        ArrayList.Add(new string[] { "7112434", "Steve", "Human Resource" });
        ArrayList.Add(new string[] { "7112435", "Trish", "Human Resource" });

I want them to convert them to separate arrays like:

        EmployeeNumber=["7112432","7112433","7112434","7112435"]
        Name =["Gagan", "Mukesh", "Steve", "Trish"]
        Department =["Human Resource", "Information Technology", "Human Resource", "Human Resource"]

I have achieved it by looping through the list using foreach but I want to know if there is any efficient way of doing this because I have like 20 million items in the original List

Moerwald
  • 10,448
  • 9
  • 43
  • 83
Gagan Deep
  • 1,508
  • 9
  • 13
  • 1
    For efficiency, you should do this where you populate the original List – Oguz Ozgul Apr 08 '20 at 10:51
  • 1
    You could use linq to transpose the list of arrays, but I doubt it will perform better. See https://stackoverflow.com/questions/39484996/rotate-transposing-a-listliststring-using-linq-c-sharp. As @OguzOzgul said, the best option is create the data with the desired structure. – Pepelui360 Apr 08 '20 at 10:57
  • Show your `foreach` loop so we can suggest improvements. Also you usually dont want to have such a big list loaded in memory, so I recommend you to think about how to process data piece by piece. – Irdis Apr 08 '20 at 11:20
  • Does this answer your question? [Rotate - Transposing a List> using LINQ C#](https://stackoverflow.com/questions/39484996/rotate-transposing-a-listliststring-using-linq-c-sharp) – Moerwald Apr 08 '20 at 11:36
  • Asking about efficiency when the ultimate goal is to end up with 3 +20.000.000 long arrays of duplicate information is rather ironic... this looks like an [XY problem](https://en.wikipedia.org/wiki/XY_problem). Why do you need those 3 arrays? What is the your goal here? – InBetween Apr 08 '20 at 11:48
  • So basically the scenario is that I have a CSV list of 6-7 fields of 20million lines. I have to save them to database , I am using Oracle.ManagedAccess to insert the bulk data to db which is quite fast (1million rows per second) but for that I need to have seperate array for each field. How can I directly convert CSV to separate field array. I am more clear now?? @InBetween – Gagan Deep Apr 08 '20 at 12:01
  • If you need to do this once, please check [SQL*Loader](https://docs.oracle.com/cd/B19306_01/server.102/b14215/ldr_concepts.htm) – Oguz Ozgul Apr 08 '20 at 12:05
  • I dont think the most efficient solution is dumping all 20.000.000 lines into memory. Maybe batching them into smaller chunks will actually make the whole process faster. And you've just proven my point; if you put all relevant information, to begin with, in your question, people wont have to guess what you are trying to do and they will probably give you better answers/advice. – InBetween Apr 08 '20 at 12:07

1 Answers1

0

This solution inspired me: Rotate - Transposing a List<List<string>> using LINQ C#

Here the code you can use for your needs:

        List<string[]> ArrayList = new List<string[]>();
        for (int i = 0; i < 20000000; i++)
        {
            //The simulation of the 20.000.000 arrays of the list takes some time (+- 10s)...
            //But you already have the list so you can skip this part of the code
            ArrayList.Add(new string[] { i.ToString(), "Gagan", "Human Resource", "AAA" });
        }

        var millis = DateTimeOffset.UtcNow.ToUnixTimeMilliseconds();

        List<List<string>> results = ArrayList.SelectMany
        (theArray => theArray.Select((itemInArray, indexInArray) => new { itemInArray, indexInArray })) //All single elements
        .GroupBy(i => i.indexInArray, i => i.itemInArray) //Group them
        .Select(g => g.ToList())
        .ToList();

        var seconds = (DateTimeOffset.UtcNow.ToUnixTimeMilliseconds() - millis) / 1000d; //9.8 s
Adrian Efford
  • 473
  • 3
  • 8
  • Yes I have. It takes a 9.8 seconds but this is NOTHING if you compare it to a foreach loop. Nonetheless you can put this in a thread or async task. So I think its a great solution – Adrian Efford Apr 08 '20 at 11:58