0

I have a problem iterating over a big DataTable (300k rows). Unfortunately, the problem isn't reproducible on demand, so I don't know how to solve it. Minimal code that should demonstrate the issue, under the right conditions:

var data = dataTable.AsEnumerable().Select(r => r.Field<string>("name")).ToArray();

The error message I see is:

System.Collections.Generic.KeyNotFoundException: The given key was not present in the dictionary.
at System.ThrowHelper.ThrowKeyNotFoundException()
at System.Collections.Generic.Dictionary`2.get_Item(TKey key)
at ___.<>c__DisplayClass10_0.<__>b__0(DataRow r) in __Database.cs:line XX
at System.Linq.Enumerable.WhereSelectEnumerableIterator`2.MoveNext()
at System.Linq.Buffer`1..ctor(IEnumerable`1 source)
at System.Linq.Enumerable.ToArray[TSource](IEnumerable`1 source)

I suspect that there is an issue with row integrity, since I see (DataRow r) in the error message. But how could the enumeration over a DataTable try to reference a row that isn't present? I suspect it isn't a column issue, since I don't see a column error message like 

Column 'name' does not belong to table .
+ System.Data.DataRow.GetDataColumn(string)
+ System.Data.DataRow.get_Item(string)
+ System.Data.DataRowExtensions.Field<T>(System.Data.DataRow, string)
+ System.Linq.Enumerable.WhereSelectEnumerableIterator<TSource, TResult>.MoveNext()
+ Buffer<TElement>..ctor(IEnumerable<TElement>)
+ System.Linq.Enumerable.ToArray<TSource>(IEnumerable<TSource>)

EDIT: The DataTable is created with this MySQL query:

using (MySqlDataAdapter dataAdapter = new MySqlDataAdapter(commandText, connectionString))
{
    dataAdapter.SelectCommand.CommandType = commandType;
    // Use DataSet to avoid bug in MySqlDataAdapter with filling tables directly
    DataSet dataSet = new DataSet();
    dataAdapter.Fill(dataSet);
    var dataTable = dataSet.Tables[0];
}
user3450049
  • 825
  • 1
  • 10
  • 20
  • When you omit `.ToArray()`, is the behavior different? Is the data being streamed somewhere? – Parrish Husband Aug 26 '18 at 01:17
  • Can we see how the DataTable is created? – Parrish Husband Aug 26 '18 at 01:32
  • The error is thrown on the .ToArray() statement. If I split it into 2 lines, one with the AsEnumerable and the second line with the .ToArray() statement, the error is thrown on the .ToArray. The DataTable is created from a MySQL query. I don't see the error every time, only ~1/200 times. – user3450049 Aug 30 '18 at 16:57

0 Answers0