Sporadic serialization failure in C#

Question

We have an Excel import into our system that we test quite rigorously. Recently, we've been noticing sporadic serialization errors.

These errors are popping up in our automated tests against the import, using the same file over and over. If we were getting this error every time, I would understand, but it seems odd that the same serialization process can fail one time and not the next.

Exception: FormatException: Input string was not in a correct format.
Stack Trace:
  at System.Number.StringToNumber(String str, NumberStyles options, NumberBuffer& number, NumberFormatInfo info, Boolean parseDecimal)
  at System.Number.ParseInt32(String s, NumberStyles style, NumberFormatInfo info)
  at System.String.System.IConvertible.ToInt32(IFormatProvider provider)
  at System.Convert.ToInt32(Object value, IFormatProvider provider)
  at System.Runtime.Serialization.Formatters.Binary.__BinaryWriter.WriteValue(InternalPrimitiveTypeE code, Object value)
  at System.Runtime.Serialization.Formatters.Binary.__BinaryWriter.WriteMember(NameInfo memberNameInfo, NameInfo typeNameInfo, Object value)
  at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.WriteKnownValueClass(NameInfo memberNameInfo, NameInfo typeNameInfo, Object data)
  at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.WriteMembers(NameInfo memberNameInfo, NameInfo memberTypeNameInfo, Object memberData, WriteObjectInfo objectInfo, NameInfo typeNameInfo, WriteObjectInfo memberObjectInfo)
  at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.WriteMemberSetup(WriteObjectInfo objectInfo, NameInfo memberNameInfo, NameInfo typeNameInfo, String memberName, Type memberType, Object memberData, WriteObjectInfo memberObjectInfo)
  at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.Write(WriteObjectInfo objectInfo, NameInfo memberNameInfo, NameInfo typeNameInfo, String[] memberNames, Type[] memberTypes, Object[] memberData, WriteObjectInfo[] memberObjectInfos)
  at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.Write(WriteObjectInfo objectInfo, NameInfo memberNameInfo, NameInfo typeNameInfo)
  at System.Runtime.Serialization.Formatters.Binary.ObjectWriter.Serialize(Object graph, Header[] inHeaders, __BinaryWriter serWriter, Boolean fCheck)
  at System.Runtime.Serialization.Formatters.Binary.BinaryFormatter.Serialize(Stream serializationStream, Object graph, Header[] headers, Boolean fCheck)

Are these unit tests? Do any of your tests modify the Excel file? — Ian Dallas, Oct 27 '11 at 17:51
These are not unit tests. These are automated in-browser tests. None of the tests modify the excel file, they just repeatedly upload them. We have also seen the error a handful of times in production, but they seem to be one-offs and users seem to be able to upload successfully if they try again. — ddango, Oct 27 '11 at 17:53
Pretty strange stack trace. Hard to imagine why the formatter thinks the member is an int32 but actually is a string. Knowing nothing about the actual object defeats a guess. — Hans Passant, Oct 27 '11 at 17:57
I would add an additional check to see, if the whole file did arrive before deserializing (length and checksum) — ralf.w., Oct 27 '11 at 18:01
re: Hans, the object graph is quite large sadly or I would post it. — ddango, Oct 27 '11 at 18:06
re: ralf, I'm pretty sure the file is arriving fine, it is only 12 lines of excel, and we're currently testing against our staging server on our local network. — ddango, Oct 27 '11 at 18:12
Is there an inner exception? Can you break in the debugger to see the value of the input string that is supposedly not in a correct format? — phoog, Oct 27 '11 at 18:26
That is what I would LIKE to try, but unfortunately we can't reproduce it reliably. — ddango, Oct 27 '11 at 18:33

score 1 · Accepted Answer · answered Oct 28 '11 at 04:03

Are you by any chance using a library that relies on reflection to map the Excel file to an object graph?

For example, I have had problems with Filehelpers corrupting data when mapping to a text file. Doesn't happen very often, but it does happen, and it is only intermittent.

In this case, the problem with FileHelpers lies in FileHelpers.RecordInfo.RecursiveGetFields(...) which in turn calls FileHelpers.FieldInfoCacheManipulator.ResetFieldInfoCache(...) which uses reflection to modify private members of the actual .NET Reflection library in an attempt to force .NET reflection to give fields back in the order they were declared.

However Microsoft explicitly states "Your code must not depend on the order in which fields/properties are returned" http://msdn.microsoft.com/en-us/library/kyaxdd3x.aspx and http://msdn.microsoft.com/en-us/library/6ztex2dc.aspx

If you are using a library that does something similar, it would explain your intermittent errors, because the library might mismatch the deserialization with the incorrect source property/field, which could be a different type.

We are using reflection to map to an intermediate set of objects that are then serialized. Interesting notes about the order of fields/properties. Through this process, I think we are realizing that the BinaryFormatter is just too brittle and prone to errors. Thinking about just XML serialization or protobuf (http://code.google.com/p/protobuf-net/) — ddango, Oct 28 '11 at 20:39

Sporadic serialization failure in C#

1 Answers1