3

I want to delete items from a CosmosDB collection using a simple query filter:

db.getCollection('RequestLog').remove({ "ApplicationId": { $eq: "myValue" } })

I get the following error:

Request rate is large. More Request Units may be needed, so no changes were made. Please retry this request later.

The RequestCharge property has the value of 46963.

The price for 40000 RUs at Azure is nearly $2400/month - which I think is way too high.

I am doing something wrong?

Why is the RequestCharge so big for a query using an index key?

  • I have an ascending index key on the ApplicationId column. The collection has about 100,000 items.
  • The collection has the Shard key value of /ApplicationId.
Catalin
  • 11,503
  • 19
  • 74
  • 147
  • You're trying to delete 100K items in a single operation? If so that is your problem. If you can amortize this operation across a greater period of time then it will require less RU/s (request units "per second") to run. – Mark Brown Mar 23 '20 at 19:09
  • @MarkBrown The entire collection has about 100K (distributed by several different `ApplicationId` values). My query should delete about 15K of them (for the specified `ApplicationId`). – Catalin Mar 23 '20 at 19:27
  • @Catalin - even a blast of 15,000 deletions could exhaust RU's (each delete costs something). You could do as Mark suggested (pacing the deletes); you could retry after the given number of "throttle" milliseconds; or you could temporarily increase RU if you really want to delete everything without being throttled (unless it's a normally-occurring operation, you don't need to increase RU for an entire month - maybe just for an hour?). – David Makogon Mar 23 '20 at 19:47
  • Yeah, my guess is that a query to return and then delete 15K items would be a fairly expensive operation. But if this is a one time operation you could scale up and then back down again (keep in mind that you can only scale back to 10% of what your max is when you start getting into this 40K range. I think if you could amortize the throughput across multiple requests and do these in batches it would be smoother for you. – Mark Brown Mar 23 '20 at 19:47
  • Thank you for input! At least I know i am not doing something stupid. Regarding the `amortize the throughput across multiple requests`: i am not sure how can I batch delete this, because I have one filter only. I will try adding a `DateCreated` filter in addition, and delete all of them, but in bulks of one day at a time. – Catalin Mar 23 '20 at 19:54
  • Sure, no problem. Other thing you want to keep an eye on is any operation needs to complete in 5 seconds on Cosmos. Another good reason to do this in smaller chunks. btw, if you regularly delete data off you may want to look at using TTL instead. Cosmos will TTL data off using unused throughput so it is as smooth an operation as you can get. Thanks! – Mark Brown Mar 24 '20 at 00:22

0 Answers0