0

I have multiple publish/subscribe servers that attempt to update a MongoDB database as events occur. Currently I am running into an issue where I need to atomically add entries to a given MongoDB document array, but am running into a race condition when it comes to who adds the entry into the document. I'm attempted to use $addToSet to accomplish the add. However, since the array entry can get updated once the entry is within the array it makes it difficult for addToSet to work properly since it does a direct comparison to see if the entry is within the array or not. The end result as shown below is I get multiple servers adding in a new entry to the array.

Below is an example of a finished document that I would like:

{
    "_id" : ObjectId("522f99a622927d6d65b81dbd"),
    "array" : [
        {
            "type" : "out",
            "uuid" : "4f501517-1d10-48c2-91b7-0ca1bda80bd5",
            "member_id" : "3847",
            "join" : ISODate("2013-09-10T17:13:58.881Z")
        }
    ],
    "created" : ISODate("2013-09-10T17:13:58.437Z"),
    "name" : "entry"
}

Below is an example of what's happening due to the race condition:

{
    "_id" : ObjectId("522f99a622927d6d65b81dbd"),
    "array" : [
        {
            "type" : "out",
            "uuid" : "4f501517-1d10-48c2-91b7-0ca1bda80bd5",
            "member_id" : "3847",
            "join" : ISODate("2013-09-10T17:13:58.881Z")
        },
        {
            "type" : "out",
            "uuid" : "4f501517-1d10-48c2-91b7-0ca1bda80bd5",
            "member_id" : "3847",
            "join" : ISODate("2013-09-10T17:13:58.881Z")
        }
    ],
    "created" : ISODate("2013-09-10T17:13:58.437Z"),
    "name" : "entry"
}

Has anyone successfully been able to atomically add/modify array entries within a document without having to move the array data out of the collection entirely? I know that by moving it out of the collection it allows me to build a unique constraint against the entry, but I'd rather not have to do that.

Patrick Santora
  • 65
  • 4
  • Operations within mongodb are always atomic at document level. Did you verify your application code for any possible bugs? For the array element to be added twice within a document, you **have to call it twice**. – Nilesh Rajani Sep 11 '13 at 16:44
  • Keep in mind that multiple servers are in play here, which is contributing to this condition. Although the document is atomic due to the nature of MongoDB, using $addToSet just isn't enough to ensure an entry goes in only if it does not exist. Mostly because the array object changes and $addToSet does not really account for that when it does its direct compare. Maybe I am missing something? – Patrick Santora Sep 11 '13 at 16:58
  • I see the point you are trying to make, but I would like to bring to your notice that mongodb does obtain [lock while performing writes](http://docs.mongodb.org/manual/faq/concurrency/). Also, [writes at document level](http://docs.mongodb.org/manual/core/write-operations/) always atomic. So while your point about $addToSet may be valid, it doesn't mean that mongoDb will execute $addToSet twice. And to have the data you have shown, it needs to call $addToSet twice. If there is nothing wrong in your application code (which I suggest you should re-verify to be sure), its a bug with mongoDB! – Nilesh Rajani Sep 11 '13 at 17:15
  • Thanks for the great feedback! However, since we have multiple servers attempting to run $addToSet almost at the same time due to the pub/sub event it does not allow MongoDB to do a proper check before adding the entry to the array. Again, most likely due to the fact that the entry's keys/values get updated after it's been initially placed into the array. – Patrick Santora Sep 11 '13 at 17:25
  • I found the following post that I have found helpful. Instead of using $addToSet I can just use $push with the proper criteria. http://stackoverflow.com/questions/4774827/mongodb-create-doc-if-not-exist-else-push-to-array – Patrick Santora Sep 11 '13 at 17:26
  • Scratch the idea of using $push. I may just need to move the array outside of it's current collection. – Patrick Santora Sep 11 '13 at 17:35

1 Answers1

0

Unfortunately, you will need to move the array into it's own collection to enforce a unique constraint against uuid, which then will cause inserts with the same uuid to fail. There is a feature request to provide that, which you can see at: https://jira.mongodb.org/browse/SERVER-1068.

Let me know if I can help you with anything else.

Best, Charlie

Charlie Page
  • 251
  • 1
  • 3