0

I seem to be unable to store a simple object to cosmos db?

this is the database model.

public class HbModel
{
    public Guid id { get; set; }

    public string FormName { get; set; }
    
    public Dictionary<string, object> Form { get; set; }
}

and this is how I store the data into the database

private static void SeedData(HbModelContext dbContext)
{
    var cosmosClient = dbContext.Database.GetCosmosClient();  
    cosmosClient.ClientOptions.AllowBulkExecution = true;

    if (dbContext.Set<HbModel>().FirstOrDefault() == null)
    {
        // No items could be picked hence try seeding.
        var container = cosmosClient.GetContainer("hb", "hb_forms");
        HbModel first = new HbModel()
       {
                Id = Guid.NewGuid(),//Guid.Parse(x["guid"] as string),
                FormName = "asda",//x["name"] as string,
                Form = new Dictionary<string, object>() // 
        }
        
        string partitionKey = await GetPartitionKey(container.Database, container.Id);
        var response = await container.CreateItemAsync(first, new PartitionKey(partitionKey));
    }
    else
    {
        Console.WriteLine("Already have data");
    }
}


private static async Task<string> GetPartitionKey(Database database, string containerName)
{
    var query = new QueryDefinition("select * from c where c.id = @id")
        .WithParameter("@id", containerName);
    using var iterator = database.GetContainerQueryIterator<ContainerProperties>(query);

    while (iterator.HasMoreResults)
    {
        foreach (var container in await iterator.ReadNextAsync())
        {
            return container.PartitionKeyPath;
        }
    }

    return null;
}

but when creating the item I get this error message

A host error has occurred during startup operation '3b06df1f-000c-4223-a374-ca1dc48d59d1'.
[2022-07-11T15:02:12.071Z] Microsoft.Azure.Cosmos.Client: Response status code does not indicate success: BadRequest (400); Substatus: 0; ActivityId: 24bac0ba-f1f7-411f-bc57-3f91110c4528; Reason: ();.
Value cannot be null. (Parameter 'provider')

no idea why it fails? the data should not be formatted incorreclty? It also fails in case there is data in the dictionary.

What is going wrong?

I am not Fat
  • 283
  • 11
  • 36

1 Answers1

2

There are several things wrong with the attached code.

  1. You are enabling Bulk but you are not following the Bulk pattern

cosmosClient.ClientOptions.AllowBulkExecution = true is being set, but you are not parallelizing work. If you are going to use Bulk, make sure you are following the documentation and creating lists of concurrent Tasks. Reference: https://learn.microsoft.com/azure/cosmos-db/sql/tutorial-sql-api-dotnet-bulk-import#step-6-populate-a-list-of-concurrent-tasks. Otherwise don't use Bulk.

  1. You are blocking threads.

The call to container.CreateItemAsync(first, new PartitionKey("/__partitionKey")).Result; is a blocking call, this can lead you to deadlocks. When using async operations (such as CreateItemAsync) please use the async/await pattern. Reference: https://github.com/davidfowl/AspNetCoreDiagnosticScenarios/blob/master/AsyncGuidance.md#avoid-using-taskresult-and-taskwait

  1. The PartitionKey parameter should be the value not the definition.

On the call container.CreateItemAsync(first, new PartitionKey("/__partitionKey")) the Partition Key (second parameter) should be the value. Assuming your container has a Partition Key Definition of /__partitionKey then your documents should have a __partitionKey property and you should pass the Value in this parameter of such property in the current document. Reference: https://learn.microsoft.com/azure/cosmos-db/sql/troubleshoot-bad-request#wrong-partition-key-value

Optionally, if your documents do not contain such a value, just remove the parameter from the call:

container.CreateItemAsync(first)

Be advised though that this solution will not scale, you need to design your database with Partitioning in mind: https://learn.microsoft.com/azure/cosmos-db/partitioning-overview#choose-partitionkey

  1. Missing id

The model has Id but Cosmos DB requires id, make sure the content of the document contains id when serialized.

Matias Quaranta
  • 13,907
  • 1
  • 22
  • 47
  • 3) not sure if I should partitionKey, but tried fetching it programmically as specified here https://stackoverflow.com/a/70757443/7400630, and parse that in await container.CreateItemAsync(first, new PartitionKey(partitionKey)); no sucess – I am not Fat Jul 11 '22 at 16:01
  • You don't need to fetch the partition key programmatically from the container. The value of the partition key should be the one in the current document. If you want to save a document that has the schema `HbModel`, what is the value of the Partition Key property (`/__partitionKey`) in that document? See the Reference I shared on step 3. All documents should have a Partition Key attribute that has the same name of the Partition Key Path or Definition, with some value. That is how partitioning works in a distributed database. – Matias Quaranta Jul 11 '22 at 16:03
  • "Partition keys By default EF Core will create containers with the partition key set to "__partitionKey" without supplying any value for it when inserting items. " - https://learn.microsoft.com/en-us/ef/core/providers/cosmos/?tabs=dotnet-core-cli – I am not Fat Jul 11 '22 at 16:21
  • 1
    If your documents do not have a property that matches the Partition Key Definition in the container, then don't pass the parameter and those documents will have an empty Partition Key: `await container.CreateItemAsync(first)`. Be advised though that this pattern won't scale in the long run, your database should be partitioned in a way that can later scale: https://learn.microsoft.com/en-us/azure/cosmos-db/partitioning-overview#choose-partitionkey – Matias Quaranta Jul 11 '22 at 16:27
  • 2
    @IamnotFat FYI this comments area is turning into a conversation, along with "play by play" results of you trying to figure out your issue. This is not what comments are for. I suggest removing all of these intermediate comments, and update your question to show what you've now done, and where your problems still lie. – David Makogon Jul 11 '22 at 17:37
  • the issue turned out to be due the jsonserializer I was using. System.Text.json does not convert the property Id to id - So I changed to using Newtonsoft instead. – I am not Fat Jul 21 '22 at 09:23