How to tell the difference between Insert and Update in Cosmos DB change feed

Question

I did get the code sample for the Cosmos DB change feed from this resource

I was able to successfully compile and run the code.

Here is the code for the change feed call

    private static async Task<ChangeFeedProcessor> StartChangeFeedProcessorAsync(
        CosmosClient cosmosClient,
        IConfiguration configuration)
    {
        string databaseName = "changefeedsample";// configuration["SourceDatabaseName"];
        string sourceContainerName = "source";// configuration["SourceContainerName"];
        string leaseContainerName = "leases";// configuration["LeasesContainerName"];

        Container leaseContainer = cosmosClient.GetContainer(databaseName, leaseContainerName);
        ChangeFeedProcessor changeFeedProcessor = cosmosClient.GetContainer(databaseName, sourceContainerName)
            .GetChangeFeedProcessorBuilder<ToDoItem>(processorName: "changeFeedSample", onChangesDelegate: HandleChangesAsync)
                .WithInstanceName("consoleHost")
                .WithLeaseContainer(leaseContainer)
                .Build();

        Console.WriteLine("Starting Change Feed Processor...");
        await changeFeedProcessor.StartAsync();
        Console.WriteLine("Change Feed Processor started.");
        return changeFeedProcessor;
    }

Here is the code for the action taken on a change feed

static async Task HandleChangesAsync(IReadOnlyCollection<ToDoItem> changes, CancellationToken cancellationToken)
        {
            Console.WriteLine("Started handling changes...");
            foreach (ToDoItem item in changes)
            {
                Console.WriteLine($"Detected operation for item with id {item.id}, created at {item.creationTime}.");
                // Simulate some asynchronous operation
                await Task.Delay(10);
            }

            Console.WriteLine("Finished handling changes.");
        }

I do see how the action can be triggered on an insert; however, is there a way to tell if there was an action taken on update. Is there a way to tell which is which, to tell one from another. Is there a way to get more details on an updated/added data

Thank you very much in advance

score 2 · Accepted Answer · answered Jul 16 '21 at 13:02

2

Is there a way to tell which is which

The CosmosDb change feed works by sending you the current value of the document. Because of this, you can't distinguish between inserts or edits.

If you only need to distinguish inserts and edits, then you can add a bool IsEdited field to your object and set it to true when the document is edited. This is sufficient for doing something like a summary/count table where you want to process inserts but ignore edits.

If you need to process edits also, then you'll need to detect multiple edits, not just the first one. In that case, you would need a second container that has the document id and last modified time (_ts), and compare the incoming _ts against the one you already saw (if any). You'll probably need a JavaScript stored procedure to do that compare+update atomically.

answered Jul 16 '21 at 13:02

Stephen Cleary

437,863
77
675
810

Another approach along the same lines that I've done is to maintain TimeCreated and TimeModified properties. At item creation both are same value, then TimeModified is changed on subsequent updates. – Noah Stahl Jul 16 '21 at 14:05
@Noah Stahl may I please ask you a different question? Is the change feed feesible tool to track the edits (value before -> value after)? Or should I use a different tool for this purpose? Thank you very much in advance for your help! – James Jul 16 '21 at 15:38
@Stephen Cleary, just in case you missed the above question, tagging here as well, thank you once again for your kind help – James Jul 16 '21 at 15:41
@Mikhail The change feed doesn't provide the previous values. If you need to audit before/after state, that would need to be done in the code that is performing the update where the old and new values are known. – Noah Stahl Jul 16 '21 at 15:46
@Mikhail - please avoid starting conversations in comments (or asking additional questions). Also, the question about whether Change Feed is a feasible tool... is really opinion-soliciting. This really isn't the place to ask that. – David Makogon Jul 16 '21 at 21:38
@Mikhail: If you want event sourcing, you need to build it. The change feed is not a full event sourcing solution. – Stephen Cleary Jul 16 '21 at 23:30

How to tell the difference between Insert and Update in Cosmos DB change feed

1 Answers1