3

Using transactional batch in cosmos, is it possible to query a partition within a collection (Select * from partition where openOrder=1) and then IF the query returns that there are no open orders, add an item to the collection.

Or you could have the write first, and then have the conditional read (Select * from partition where openOrder=1 and id={unique GUID just written}) and then if the 2nd read fails, the write would be reversed (?).

Would want this to be done in one atomic operation because I don't want to think there are no open orders and then another process writes an open order before this process can.

If not, is it possible to do this a different way?

**Edit 3:30 PM PT 9/16 with attempted solution that wrote a document when I tested it **

function checkOpenOrder(inputDocString, query){
    console.log("Stored Procedure Starting")
    var context = getContext(); 
    var container = context.getCollection();
    var containerLink = container.getSelfLink();
    var response = context.getResponse(); 

    var isAccepted = container.queryDocuments(
        container.getSelfLink(), 
        query,
        function (err, items, options) {
            if (err) throw err;
            // Query would be fed in such that if there is no open order, no items would return in the collection 

            if (items.length == 0){ 
                var docCreated = container.createDocument(containerLink, inputDocString, 
                    function (err2, itemWritten) {
                        if (err2) throw err2;
                        // else was successfully able to write document?
                        response.setBody("Wrote document");
                    });

            }
            else { 
               
                response.setBody("Order currently open");
            }
            

        });
    if (!isAccepted) throw new Error('The query was not accepted by the server.')

    
}

Edit: Final question 9/17 Would I want to set max degree of parallelism to 1 to ensure that the stored procedure cannot run 2x in the same partition at the same time? (which could create a race condition if it sees there's no open order -> creates a document and then we have 2 open orders). I think that I want cross partition query disabled (plus I will not need it). enter image description here

WIT
  • 1,043
  • 2
  • 15
  • 32
  • 1
    I wonder if you could use the ETag of a "sentinel" record to do this. The sentinel would be an alternative representation of the latest state of your `select *` result. Then for an order creation to succeed, you'd need also simultaneously to update the sentinel record using its ETag. Only the first writer would succeed, the second would get a 412. This assumes such a sentinel item could be maintained for each partition to replace the scan query. – Noah Stahl Sep 16 '20 at 00:18

2 Answers2

1

I am afraid the answer is no. TransactionalBatch only supports write operations. If you want a cross-document read-write transaction as in your case, you only have two options:

  1. Do the transaction in a stored procedure. This only works if you are working within a single partition, and you do not need to perform any non-Cosmos-related operations within your transaction. From your description, this should be ok.

  2. Implement the transaction on the client side with some kind of locking.

Mo B.
  • 5,307
  • 3
  • 25
  • 42
  • I am working within a single partition. Ideally if the write does not occur (bc the query comes back with openOrder=1) then I would return either a certain message (or error) that the application would parse. So is it sort of something exactly like this - https://stackoverflow.com/questions/54443220/can-i-check-for-uniqueness-and-write-a-document-in-a-cosmos-db-stored-procedure? – WIT Sep 15 '20 at 21:47
  • No, it's not exactly the same issue, because you are not dealing with a single unique document. But you can write a server-side stored procedure that does exactly what you need: Check if an open order exists, and only if not, create the new order, and return a boolean indicating whether a new order was created. This will run as a transaction. – Mo B. Sep 15 '20 at 21:59
  • I attempted an approach based on a stored procedure but I am not very familiar with javascript so tried to follow examples, if you have a chance would it be possible to see if the proposed solution would work? – WIT Sep 16 '20 at 15:02
  • @MoB. just one comment, TransactionalBatch does not only support write operations https://learn.microsoft.com/dotnet/api/microsoft.azure.cosmos.transactionalbatch.readitem?view=azure-dotnet – Matias Quaranta Sep 16 '20 at 15:12
  • 1
    @MatiasQuaranta Yes, you're right, but it's extremely limited. AFAIK you can't use the read item inside the transaction, so it's not useful for common read-write transactions where the write is based on properties from the read item. And you can't issue a query within a TransactionBatch. – Mo B. Sep 16 '20 at 15:20
  • @WIT Feel free to update the question with your attempt. – Mo B. Sep 16 '20 at 15:24
  • @MoB. I updated the question with the attemptedt this AM, were you able to see it? – WIT Sep 16 '20 at 17:27
  • this is one of my first javascript functions and it is my first cosmos stored procedure – WIT Sep 16 '20 at 17:28
  • @MoB. I was actually able to attempt again and I think I have a working solution, I am not 100% sure that it works as expected if you had a chance to take a look, but I think it should – WIT Sep 16 '20 at 22:31
  • @WIT Looks right. You should test it by running it in parallel, with each task attempting to create a different document, and asserting that exactly one document has been written after all tasks completed. – Mo B. Sep 17 '20 at 06:57
  • @MoB. I added one more clarification regarding whether I should set max degree of parallelism to 1. Apologies if it sounds stupid/for all of the notifications, this is my first stored proc. – WIT Sep 17 '20 at 14:40
  • Stored procedures are guaranteed to be transactional no matter what the settings are. You can run them in parallel. – Mo B. Sep 18 '20 at 07:40
  • @MoB. but if 2 orders for the same customer run at the same time, then you could have a race condition if they run in parallel right? – WIT Sep 18 '20 at 15:11
  • @WIT No, that's the whole point of using stored procedures in Cosmos DB. They run with ACID guarantees. I thought that was also the point of your question in the first place - how to avoid race conditions. – Mo B. Sep 18 '20 at 19:37
  • @MoB. it was, so that means if 2 separate requests come in for the same customer Id (where customer ID is the partition), they will run sequentially, even if I set max degree of parallelism to > 1? – WIT Sep 21 '20 at 22:36
  • I know that ACID guarantees are within 1 run of the stored procedure, but I wasn't sure if the same stored procedure is triggered 2x (or more) at a time for a partition and they both read the same value at the same time and then wrote at the same time. – WIT Sep 21 '20 at 22:37
  • Continuing the discussion in chat: https://chat.stackoverflow.com/rooms/221640/discussion-between-wit-and-mo-b – Mo B. Sep 22 '20 at 07:49
1

As mentioned in one of the answers, the TransactionBatch only supports write operations. We had faced a similar challenge with Cosmos DB. Here's how you could approach the problem:

  • You can introduce a concept of "Session" where Commit/Save operation happens at the end of request/ session.
  • Create a local identity map/ cache for all the Get operations. That is, first check if the Cosmos document is present in the local cache, if yes, then return it from local cache. Else, fetch it from Cosmos.
  • The Create, Upsert or Delete operation only saves to the local cache and updates the identity map.
  • The Commit operation is called at the end of the request. It uses the items in the local cache to create a transaction batch and save it to Cosmos.

The advantage of this approach is that you can do all this in C# without any complex locking code.

A couple of limitations of this approach are:

  • It can only work within the single partition (which I think is fine for you)
  • Since you are working with ids of Cosmos Document, it can tricky to pass a custom QueryDefinition which does not work with ids.
Ankit Vijay
  • 3,752
  • 4
  • 30
  • 53
  • You will still need client-side locking to avoid the read-write race condition the OP is concerned about. – Mo B. Sep 16 '20 at 09:28