How do I conditionally upsert a document in mongo?

Question

I've got a document

{ key : 'key1', value : 'value1', update_time : 100 }

That I'd like to change only with more recent (greater) update times. What I'm doing now is:

def update_key1(new_value, new_time):
    record = find_one( { key : 'key1' } )
    if not record or record['update_time'] < new_time:
        update( { key : 'key1', value : new_value, update_time : new_time }, upsert=True)

Obviously this is an extra roundtrip to the db, but more importantly there's no lock on the document, and concurrent calls could result in a lower new_time's value remaining in the db. Is there a way to perform an upsert only if a condition is true?

EDIT: Just to clarify, the intention is not to create multiple documents for each key and then sort on lookup. Though that would solve my problem, these values change a lot and would waste a lot of space.

I have had the similar question! http://stackoverflow.com/questions/10059739/mongodb-insert-on-duplicate-key-update — Ali Shakiba, Feb 14 '13 at 09:17
Have you been able to solve this issue? I am having exactly similar use case and I am not able to solve it even after using $setOnInsert. — ameykpatil, Jul 08 '13 at 10:34

score 4 · Answer 1 · answered Oct 15 '20 at 16:57

If you add a unique constraint on key, then updateOne with an update_time filter and upsert=True will

insert missing records
update stale records
throw an error when you try to update using stale input (because the update will not match the filter condition, and the insert will fail due to the constraint)

collection.updateOne({'key': key,
                      'update_time': {'$lt': new_time}
                      },
                     {'$set': {'value': new_value,
                               'update_time': new_time
                               }
                      },
                     upsert=True
                     )

You can catch errors and check for code: 11000 (unique constraint violations) to specifically handle those cases of having tried to update to a past state.

If you can have uniqueness set on the key in question, this works well. — levigroker, Jun 10 '21 at 17:40

rakslice · Answer 2 · 2013-10-06T23:26:22.980

Short of being able to do the whole thing atomically, there are two kinds of existing conditions where you want to make a change, and you can deal with each of them atomically:

no record for the key exists
a record for the key exists and its update_time is older than new_time

Update an existing record for key:

def update_if_stale(key, new_value, new_time):
    collection.update({'key': key,
                       'update_time': {'$lt': new_time}
                       },
                      {'$set': {'value': new_value,
                                'update_time': new_time
                                }
                       }
                      )

Insert if a record for key didn't exist before:

def insert_if_missing(key, new_value, new_time):
    collection.update({'key': key},
                      {'$setOnInsert': {'value': new_value,
                                        'update_time': new_time
                                        }
                       },
                      upsert=True
                      )

($setOnInsert was added in MongoDB 2.4)

You might be able to put those together to get what you need, e.g.:

def update_key(key, new_value, new_time):
    insert_if_missing(key, new_value, new_time)        
    update_if_stale(key, new_value, new_time)

However, depending on what remove/insert time scales might be possible in your system, you might need multiple calls (update/insert/update) or other shenanigans.

Aside: If you want a record missing the update_time field to be treated as a stale record to update, change {'$lt': new_time}} to {'$not': {'$gte': new_time}}

score 1 · Accepted Answer · answered Dec 07 '12 at 19:50

1

If WPCoder's response with and upsert=True isn't what you're looking for, than you may need $setOnInsert, which isn't implemented yet: https://jira.mongodb.org/browse/SERVER-340. It should be implemented for 2.4.0.

answered Dec 07 '12 at 19:50

kris

23,024
10
70
79

score 0 · Answer 4 · answered Dec 07 '12 at 15:34

0

Seems that you're looking for findAndModify. Sorry, I'm not familiar with python so I won't be able to give you a code snippet but the command should be pretty straight forward.

answered Dec 07 '12 at 15:34

karka91

733
1
6
20

I don't think findAndModify will solve this. The issue arises from identifying whether to perform a modification on a record. I don't see a way to do that from the docs. – Parker Dec 07 '12 at 16:12

score 0 · Answer 5 · answered Dec 07 '12 at 19:13

0

You can do a conditional update:

db.MyObjects.update( { key : "key1", value: { $lt : aValue }},
                    { $set : { value : aValue }});

answered Dec 07 '12 at 19:13

WiredPrairie

58,954
17
116
143

But I'd like to insert the object if it doesn't exist already. And if I change this to an upsert, it will add a new document if it's not the newest, right? – Parker Dec 11 '12 at 18:40
Yes, it should work that way if you're query is correct. Make sure your _id column is provided or it will create a new document. – WiredPrairie Dec 11 '12 at 21:41
@WiredPrairie The reason to do an upsert is because you don't know if a matching document exists; how could you know its _id to provide? – rakslice Oct 06 '13 at 21:36
You don't need to use an `ObjectId` for an `_id`. – WiredPrairie Oct 07 '13 at 01:02
@WiredPrairie Your suggestion works but it results in "duplicate key error collection". Is there any cleaner way to do this? – user1870400 Sep 08 '17 at 10:35
@Parker How did you end up solving this? – user1870400 Sep 08 '17 at 10:35
@user1870400 - I'd suggest asking a new question. This one is almost 5 years old and may not represent current best practices. – WiredPrairie Sep 08 '17 at 17:42
@WiredPrairie If I ask another question similar to this it will be tagged as Duplicate. – user1870400 Sep 08 '17 at 19:16
@user1870400-there isn't an accepted answer, so it can't be closed as a dup. You'll need to explain your scenario and what's not working. – WiredPrairie Sep 13 '17 at 10:33
As written, this answer is unhelpful because it does not answer the question. Related comments suggest this "works" only if one includes the `_id` field. The inclusion of the `_id` field makes this work because this field is created by mongo and is "unique" but this technique is better described by @ejohnson's answer. – levigroker Jun 10 '21 at 17:44

How do I conditionally upsert a document in mongo?

5 Answers5

Linked