1

If I set TTL of a document in azure cosmos db to be 1 sec, then it never reaches changefeed What is the reason for that? Most probably, before the change diff can be computed the data is deleted.

If that's the reason, what should be the minimum value of TTL, which guarantees that it will land up in change feed ?

biswpo
  • 771
  • 7
  • 23

1 Answers1

2

Setting the TTL to 1 second it's a Replace operation. It will appear on the change feed. The key there is, you need to have something in place (Change Feed Processor, Azure Functions) that would react in less than 1 second (read the change and act on it), otherwise it will miss it.

So the amount of seconds to set on the TTL is equal to the amount of seconds it takes to your detection mechanism to pick it up.

Having said that, you can actually turn around the scenario. Why not add a "delete" flag on the document (soft delete), and when it gets picked up by your Change Feed detection mechanism, set the TTL to 0 on it? That would first let you detect the deletion intent and act on it, and then, setting the TTL to 0 would delete the document without a second Change Feed notification.

Matias Quaranta
  • 13,907
  • 1
  • 22
  • 47
  • as per my understanding TTL set to 1sec will delete the document from collection and deleted document will not come to change feed. And that is why to get deleted items in changefeed the alternative was to add soft-delete flag and set TTL to remove from change feed. So I don't get it are you saying even deleted document will get captured in changeFeed ?? – Abhishek Feb 07 '20 at 10:59
  • 1
    I don't say a deleted document will get captured, what I say is, the operation of setting the TTL is an Update. That update operation will be in the Change Feed for 1 second, until the document is deleted. If you read the Change Feed every 500ms, you will detect the Update operation, but if you read the Change Feed every 5 seconds, you won't. That is why I mention that it depends on the interval you are using. – Matias Quaranta Feb 07 '20 at 16:29
  • @MatiasQuaranta we have changefeed processor listening to the changefeeds, but we tested locally a ttl update of 1 sec doesn't reach the changefeed,is there any parameter I need to set ? – biswpo Feb 19 '20 at 09:57
  • @biswpo Again, this depends on how often your Change Feed Processor is checking for changes. And if it's currently not processing anything. Please see https://learn.microsoft.com/en-us/azure/cosmos-db/change-feed-processor#processing-life-cycle. I would not rely on the CFP to be checking the changes before they disappear, the host could be restarting, or processing other changes, or simply sleeping the default customizable 5 seconds. That is why the recommendation is to add the soft delete. – Matias Quaranta Feb 19 '20 at 16:10