0

I convert a DataSet from a SQL Server query where the double values are rounded by the SQL Server to two decimal digits. After converting to json with

json = JsonConvert.SerializeObject(ds(1), New DataSetConverter());

some (very few) values suddenly have 16 decimal digits. Is there a way to have this conversion to avoid this (rounding, fixed no of decimals etc.)?

Bruno Ribeiro
  • 5,956
  • 5
  • 39
  • 48
Peo
  • 11
  • 2
  • The problem is that there base 10 decimals cannot always be represented using finite precision as base 2 decimals - and `double` is a base 2 decimal. This causes tiny precision loss when round-tripping doubles to strings. See [JObject.Parse modifies end of floating point values](http://stackoverflow.com/questions/26484176) for a possible workaround. I an not 100% sure that `DataSetConverter` checks [`FloatParseHandling`](http://www.newtonsoft.com/json/help/html/P_Newtonsoft_Json_JsonSerializerSettings_FloatParseHandling.htm) though. – dbc Feb 10 '16 at 16:34
  • It may be that the precision loss occurred when deserializing the SQL data to `double`. From the [docs](https://msdn.microsoft.com/en-us/library/system.double%28v=vs.110%29.aspx): *A value might not round-trip if a floating-point number is involved.* Can use you `decimal` in your `DataSet` instead? – dbc Feb 10 '16 at 16:37
  • @bruno I am aware of the representation problem (I should have mentioned that) but this happens with values that are possible to represent without this problem, example 2.97 is after conversion to json 2.96999999999999998. When I test to assign 2.97 to a double (or a single for that matter) or parse a string of "2.97" to a double it still has a value of 2.97 – Peo Feb 12 '16 at 09:26

0 Answers0