1

I'm using Azure APIM policy expression to aggregate multiple responses. I have some decimal values in the response. But while Deserializing , formatting was changed as shown in the output. I want to return as in the Input.

INPUT

{
    "x1": 1.55391E4,
    "x2": 2.2173244E5,
    "x3": 1.11226E3,
    "UpdatedDateTime": "2023-01-17T20:45:51.959+08:00"
}

OUTPUT

{
    "x1": 15539.1,
    "x2": 221732.44,
    "x3": 1112.26,
    "UpdatedDateTime": "2023-01-17T20:45:51.959+08:00"
}

EXPECTED

{
    "x1": 1.55391E4,
    "x2": 2.2173244E5,
    "x3": 1.11226E3,
    "UpdatedDateTime": "2023-01-17T20:45:51.959+08:00"
}

This is my fiddle

In this sample, I have preserved the DateTimeZone with Offset. but I can't do the decimal fields (x1, x2, x3). I just wants to return as it is like input.

Please note that I'm writing this inside a policy expression, so I can't create any C# extensions or helper methods.

Abdul Wahab
  • 169
  • 10
  • You can't preserve the original decimal formatting with Json.NET. When JsonTextReader encounters a floating point JSON number, it parses it to `decimal` (or `double`) **and discards the original JSON character sequence**. Thus only the value (and number of digits in the case of `decimal`) is retained. – dbc Jan 17 '23 at 15:02
  • 1
    Utf8JsonReader from System.Text.Json, on the other hand, **does** retain the underlying JSON character sequence. And this character sequence is passed off to `JsonElement` and `JsonNode` which also retain the original character sequence and present read-only (or editable) views of it. So if you switch to System.Text.Json you willl be able to retain the precise decimal formatting. Demo here: https://dotnetfiddle.net/wxqwgt. But can you switch, or do you need to stick with Json.NET? And if you can switch, what .NET version are you working in? – dbc Jan 17 '23 at 15:05
  • 1
    @dbc, I believe Azure APIM policy expressions don't support System.Text.Json, Allowed CLRTypes: https://learn.microsoft.com/en-us/azure/api-management/api-management-policy-expressions#CLRTypes. I can't switch – Abdul Wahab Jan 17 '23 at 15:32
  • That's unfortunate. Do you need to preserve the decimal format, or would it be enough to force scientific notation? – dbc Jan 17 '23 at 21:55
  • I need to force scientific notation, preferably this 2.2173244E5 from 221732.44 – Abdul Wahab Jan 18 '23 at 02:35
  • I devised a workaround by converting the JObject to decimal and applying the necessary formatting. This is not a one-size-fits-all solution. If new decimal fields are added to the backend API. I need to go over my policy expressions again and mention the each individual JProperty. I also considered scanning the JValue with a RegEx approach. Please let me know if you have any generic solutions. https://dotnetfiddle.net/TuXrO8 – Abdul Wahab Jan 18 '23 at 03:37
  • 1
    All I can think to do is to recursively descend the `JToken` hierarchy and replace all `decimal` valued `JValue` tokens with a `JRaw` with the required format. (I checked that `JRaw` is allowed as per https://learn.microsoft.com/en-us/azure/api-management/api-management-policy-expressions#CLRTypes). See https://dotnetfiddle.net/cTsQOt. Is that any better? – dbc Jan 18 '23 at 03:53
  • 1
    @dbc, This one is better than my approach, please add this as answer. Thank you! – Abdul Wahab Jan 18 '23 at 04:13

1 Answers1

0

One way to force scientific notation for decimal values in a JToken hierarchy would be to replace decimal-valued JValue tokens with an appropriately formatted JRaw value:

var settings = new JsonSerializerSettings 
{
    // Make sure that FloatParseHandling is consistent with the later check ".Where(v => v.Value is decimal)"
    FloatParseHandling = FloatParseHandling.Decimal, 
    FloatFormatHandling = FloatFormatHandling.DefaultValue, 
    // Instead of DateParseHandling.DateTimeOffset, you could use DateParseHandling.None to skip DateTime recognition and leave date/time strings unchanged.
    DateParseHandling = DateParseHandling.DateTimeOffset, 
    DateTimeZoneHandling = DateTimeZoneHandling.Unspecified
};

var obj = JsonConvert.DeserializeObject<JObject>(json, settings);

var decimalValues = obj.Descendants().OfType<JValue>().Where(v => v.Value is decimal).ToList(); 
foreach (var value in decimalValues)
{
    value.Replace(new JRaw(((decimal)value.Value).ToString("0.00000E0" /*, System.Globalization.CultureInfo.InvariantCulture */))); // Is System.Globalization.CultureInfo.InvariantCulture available?
}

var newJson = obj.ToString(Formatting.Indented);

Which results in

{
  "x1": 1.55391E4,
  "x2": 2.21732E5,
  "x3": 1.11226E3,
  "UpdatedDateTime": "2023-01-17T20:45:51.959+08:00"
}

Demo fiddle #1 here.

Notes:

  • You code is inside an Azure APIM policy expression. Only a very limited set of types are allowed in such an expression, as documented in .NET Framework types allowed in policy expressions. Of note, the following are not available:

    • Newtonsoft.Json.JsonConverter.
    • Newtonsoft.Json.JsonTextReader and JsonTextWriter.
    • System.Text.Json (all).

    The lack of any ability to create a custom converter is why I suggested to use JRaw.

  • You can't preserve the original decimal formatting with Json.NET. When JsonTextReader encounters a floating point JSON number, it parses it to decimal or double and discards the original JSON character sequence. Thus only the value (and number of digits in the case of decimal) is retained.

  • Utf8JsonReader from System.Text.Json, on the other hand, does retain the underlying JSON character sequence. This character sequence is passed off to JsonElement and JsonNode which also retain the original character sequence and present read-only (or editable) views of it. So if Azure APIM policy expressions are ever enhanced to allow System.Text.Json you would be able to retain the original JSON decimal formatting much more easily.

    Demo fiddle #2 here.

  • If you want to leave all date & time string values unchanged, instead of DateParseHandling.DateTimeOffset, you could use DateParseHandling.None to disable DateTime recognition entirely.

dbc
  • 104,963
  • 20
  • 228
  • 340