Cosmos DB Serialization Issue Throws Error Reading An Int as Double

Question

On some of my documents my .NET Cosmos Db client (v3) will not read a document(s). The document has a property DistanceInMetres and is set to an int on my class.

   public int DistanceInMetres { get; set; }

When I examine one of the documents causing this error (there are numerous) the property is set correctly.

   "distanceInMetres": 3272,

The error I receive is:

 Input string '3272.0' is not a valid integer. Path 'distanceInMetres', line 1, position 1586.

This is the code that does the request:

    public async Task<T> ReadEntityAsync<T>(string id, string pKey)
    {
       var doc= await container.ReadItemAsync<T>(id, new PartitionKey(pKey));
       return doc;
    }

The serializion error is not depenedent on the client query it occurs on a Container.Read as well as a Container.Query for multiple documents. I tried changing the underlying model property from int to double I still get the same error! I've checked other documents with the same model that return with out the error to see if there is any difference but there isn't (That I can see). If I go into the portal and change the property manually while keeping the value the same and update the document the error goes. I don't believe it's a problem with the data access it happens over single and multiple queries. Is it a bug on CosmosDB? Has any one any ideas?

As an update: When I use the DocumentClient to access the database:

 DocumentClient documentClient = new DocumentClient(new 
 Uri("url"),"Key",                                                         
 serializerSettings: new JsonSerializerSettings { ContractResolver = new 
 CamelCasePropertyNamesContractResolver() }) ;

The document is read with no exceptions.

When I use CosmosClient:

   CosmosClient cosmo = new 
   CosmosClient(config["keys:cosmosClientConStr"], new CosmosClientOptions
   {
       ConnectionMode = ConnectionMode.Direct,
       SerializerOptions = new CosmosSerializationOptions { 
       PropertyNamingPolicy = CosmosPropertyNamingPolicy.CamelCase }
        });

        Container container = cosmo.GetContainer(config["keys:dbName"], 
 config["keys:colName"]);

I get the exception on SOME documents. The strange thing is if I go into the portal and re-save the document everything is ok. UPDATE: If I change the class type from T to Dynamic I can read the item back and it is indeed read as a decimal even though it's stored as a integer. Here is the stack trace.

   at Newtonsoft.Json.JsonTextReader.ParseReadNumber(ReadType readType, Char firstChar, Int32 initialPosition)
   at Newtonsoft.Json.JsonTextReader.ParseNumber(ReadType readType)
   at Newtonsoft.Json.JsonTextReader.ReadNumberValue(ReadType readType)
   at Newtonsoft.Json.JsonTextReader.ReadAsInt32()
   at Newtonsoft.Json.JsonReader.ReadForType(JsonContract contract, Boolean hasConverter)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.PopulateObject(Object newObject, JsonReader reader, JsonObjectContract contract, JsonProperty member, String id)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateObject(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.CreateValueInternal(JsonReader reader, Type objectType, JsonContract contract, JsonProperty member, JsonContainerContract containerContract, JsonProperty containerMember, Object existingValue)
   at Newtonsoft.Json.Serialization.JsonSerializerInternalReader.Deserialize(JsonReader reader, Type objectType, Boolean checkAdditionalContent)
   at Newtonsoft.Json.JsonSerializer.DeserializeInternal(JsonReader reader, Type objectType)
   at Newtonsoft.Json.JsonSerializer.Deserialize(JsonReader reader, Type objectType)
   at Newtonsoft.Json.JsonSerializer.Deserialize[T](JsonReader reader)
   at Microsoft.Azure.Cosmos.CosmosJsonDotNetSerializer.FromStream[T](Stream stream)
   at Microsoft.Azure.Cosmos.CosmosJsonSerializerWrapper.FromStream[T](Stream stream)
   at Microsoft.Azure.Cosmos.CosmosJsonSerializerWrapper.FromStream[T](Stream stream)
   at Microsoft.Azure.Cosmos.CosmosSerializerCore.FromStream[T](Stream stream)
   at Microsoft.Azure.Cosmos.CosmosResponseFactoryCore.ToObjectpublic[T](ResponseMessage responseMessage)
   at Microsoft.Azure.Cosmos.CosmosResponseFactoryCore.<CreateItemResponse>b__8_0[T](ResponseMessage cosmosResponseMessage)
   at Microsoft.Azure.Cosmos.CosmosResponseFactoryCore.ProcessMessage[T](ResponseMessage responseMessage, Func`2 createResponse)
   at Microsoft.Azure.Cosmos.CosmosResponseFactoryCore.CreateItemResponse[T](ResponseMessage responseMessage)
   at Microsoft.Azure.Cosmos.ContainerCore.<ReadItemAsync>d__56`1.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.ConfiguredTaskAwaitable`1.ConfiguredTaskAwaiter.GetResult()
   at Microsoft.Azure.Cosmos.ClientContextCore.<RunWithDiagnosticsHelperAsync>d__38`1.MoveNext()
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at CT5ApiService.ApiData.CosmosStore.<ReadEntityAsync>d__4`1.MoveNext() in C:\Users\pjsta\source\repos\CT_CLOUD\CT5ApiService\ApiData\CosmosStore.cs:line 81
   at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw()
   at System.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccessAndDebuggerNotification(Task task)
   at System.Runtime.CompilerServices.TaskAwaiter`1.GetResult()
   at CT5ApiService.Api.JobsApi.<Get>d__3.MoveNext() in C:\Users\pjsta\source\repos\CT_CLOUD\CT5ApiService\Api\JobsApi.cs:line 49
       ````

Are all documents created the same way? Or is the issue restricted to documents created by a specific user/process/client? — DeMaki, Jan 28 '21 at 13:02
Seems very odd that the property is not a decimal in the database and is stored as an integer. — Paul Stanley, Jan 28 '21 at 13:45
No I not using any culture the db is in UK South and so am I. — Paul Stanley, Jan 28 '21 at 16:09
It doesn't seem that you have provided the actual code that does the query operation... — Noah Stahl, Jan 28 '21 at 22:44

score 1 · Answer 1 · answered Jan 29 '21 at 18:44

Given your comment:

if I go into the portal and re-save the document everything is ok

This suggests whatever last modified the item is likely introducing some improper encoding that isn't a true UTF-8 integer value. Your re-save corrects the encoding. Hidden characters can cause serialization problems.

A more robust approach is to use the ReadItemStreamAsync SDK method to read as a raw Stream, then convert to an item using either System.Text.Json.JsonSerializer.DeserializeAsync or JsonConvert on the stream. Then you have opportunity to inspect the raw value and handle any exceptions that arise.

An example is show in the samples repo:

using (ResponseMessage responseMessage = await container.ReadItemStreamAsync(
    partitionKey: new PartitionKey("Account1"),
    id: "SalesOrder1"))
{
    // Item stream operations do not throw exceptions for better performance
    if (responseMessage.IsSuccessStatusCode)
    {
        SalesOrder streamResponse = FromStream<SalesOrder>(responseMessage.Content);
        Console.WriteLine($"\n1.2.2 - Item Read {streamResponse.Id}");

        // Log the diagnostics
        Console.WriteLine($"\n1.2.2 - Item Read Diagnostics: {responseMessage.Diagnostics.ToString()}");
    }
    else
    {
        Console.WriteLine($"Read item from stream failed. Status code: {responseMessage.StatusCode} Message: {responseMessage.ErrorMessage}");
    }
}

Unfortunetely this line throw the same error. - .return Serializer.Deserialize(jsonTextReader); However, I think your right there is something amiss when the document is saved. The exception is not thrown using DocumentClient so I will have to implement my class with that instead. — Paul Stanley, Jan 29 '21 at 22:06

Cosmos DB Serialization Issue Throws Error Reading An Int as Double

1 Answers1