Converting 1000s ExpandoOject / dynamic to static type takes forever

Question

I am trying to convert a list of 1000s dynamic (that I get from CsvHelper reading csv file) into static type but its taking forever.

Here's code:

dynamic object

      MyObj {
               id =1, prop1=1,prop2=2,prop3=3...
            }

result

           PObj1 { oid = 1 , name = "Property 1", value = "1" }
           PObj2 { oid = 1 , name = "Property 2", value = "2" }
           PObj3 { oid = 1 , name = "Property 3", value = "3" }

Code to convert

 var imp = rows.SelectMany(x => map.Keys.ToList().Select(k => new PObj
      {
          OID = (((IDictionary<string, object>)x)["oid"] ?? "").ToString(),
          Name = k,
          Value = ToDate((((IDictionary<string, object>)x)[map[k]] ?? "").ToString())
      }).ToList()).ToList();

map contains list of properties about 40-50

map<string,string>{
                   {"Property 1","prop1"},
                   {"Property 1","prop2"},
                   {"Property 1","prop3"}
                    ...
}

ToDate function

private DateTime? ToDate(string strDate)
        {
            strDate = strDate.Split(' ')[0];
            strDate = strDate.Replace('-', '/').Replace('.', '/');
            DateTime? dt = null;
            try
            {
                dt = DateTime.ParseExact(strDate, dateFormats, CultureInfo.InvariantCulture, DateTimeStyles.None);
            } catch { }
            return dt;
        }

map can contain any number of peroperties hence expandoObject will have dynamic number of properties.

Is there any way I can improve the performance?

The reason I need to do this conversion is because I need to than send this as table to a stored procedure therefore converting expandoObject straight into table creates issue if number properties in object changes as this mean number of column will also change in table.

I am open to other solutions as well if works in above situation.

Check Task Manager while code is running and see what the memory usage is. Linq objects/methods use lots of memory and slow execution time is often an issue with memory. — jdweng, Dec 13 '17 at 07:58
Change the ParseExact into a TryParseExact and see if that makes a difference. Suppose ParseExact fails 90% of the time then it means you're generating a lot of exceptions, which are performance killers — Joost Aarts, Dec 13 '17 at 08:44
@JoostAarts that didn't make any difference. BTW its revers, it succeeds almost 95% of the times — Bhavesh, Dec 13 '17 at 09:54
I must be doing something wrong here, its only about 15k dynamic object with 8 properties which should create about 130k objects which is not big number but still its taking forever — Bhavesh, Dec 13 '17 at 09:59
not sure how but same code (almost same) now works fine. it completes whole list in less than second. rows.ToList().ForEach(x => imps.AddRange(map.Keys.Select(k => new ImportMilestone { JVSiteID = (((IDictionary)x)[siteid] ?? "").ToString(), Milestone = k, MilestoneValue = ToDate((((IDictionary)x)[map[k]] ?? "").ToString()) }).ToList())); — Bhavesh, Dec 13 '17 at 10:14

score 0 · Answer 1 · answered Dec 13 '17 at 10:16

seems like it was my pc (running windows on mac). Same code now works fine

 rows.ToList().ForEach(x => imps.AddRange(map.Keys.Select(k => new ImportMilestone
                    {
                        JVSiteID = (((IDictionary<string, object>)x)[siteid] ?? "").ToString(),
                        Milestone = k,
                        MilestoneValue = ToDate((((IDictionary<string, object>)x)[map[k]] ?? "").ToString())
                    }).ToList()));

Converting 1000s ExpandoOject / dynamic to static type takes forever

1 Answers1