AWS DocumentDB Syntax differs from MongoDB for UpdateMany

Question

I'm attempting to perform an update using a substr after migrating from mongo > documentDB, however I am getting a strange error.

The following works on Mongo 4.0 using pymongo:

    await db[cls.colname].update_many(
        {'$or': [{'preview_title': {'$exists': False}}, {'preview_title': None}]},
        [{'$set': {'preview_title': {'$substr': ['$title', 0, 70]}}}],
    )

The equivalent mongo shell command also works:

db.getCollection('pages').updateMany({'$or': [{'preview_title': {'$exists': false}}, {'preview_title': null}]}, [{'$set': {'preview_title': {'$substrBytes': ['$title', 0, 70]}}}])

However, after switching to AWS DocumentDB, it doesn't accept the update as an array or it will return this error: "MongoError: Wrong type for parameter u" image

If I change the command to the below, it will work (basically just remove the square brackets from the update portion):

db.getCollection('pages').updateMany({'$or': [{'preview_title': {'$exists': false}}, {'preview_title': null}]}, {'$set': {'preview_title': {'$substr': ['$title', 0, 70]}}})

This would be fine, however, the equivalent pymongo call doesnt seem to work:

await db[cls.colname].update_many(
            {'$or': [{'preview_title': {'$exists': False}}, {'preview_title': None}]},
            {'$set': {'preview_title': {'$substr': ['$title', 0, 70]}}},
        )

It returns the following error: pymongo.errors.WriteError: Document can't have $ prefix field names: $substr

D. SM · Accepted Answer · 2021-03-16T22:59:17.143

2

Passing array as the update is called "updating with an aggregation pipeline". DocumentDB apparently doesn't implement that (among a bunch of other MongoDB features).

When you remove the array from update you change from using https://docs.mongodb.com/manual/reference/operator/aggregation/set/ to using https://docs.mongodb.com/manual/reference/operator/update/set/. You'll notice that the update $set doesn't accept expressions like aggregation $set does. So your shell update with $set writes a (nested) document {'$substr': ['$title', 0, 70]} into the preview field. Generally keys starting with dollars are not queryable because the dollar prefix is used by MongoDB operators, so writing such a document into a field is a bad idea. Pymongo tells you this and refuses the operation. The shell allows it because it's also used internally by MongoDB server engineers for testing and so it permits various weird/unusual things to be done.

edited Mar 16 '21 at 22:59

answered Mar 16 '21 at 13:17

D. SM

13,584
3
12
21

I did see that, thats why I changed the approach as per the second half of the question. What is throwing me off is the fact that seemingly the same statement is working on DocDB using mongo shell, but not working using pymongo. – ABroz Mar 16 '21 at 21:48
Updated the answer. – D. SM Mar 16 '21 at 22:59
Got it, I'll go for a different approach. Thanks! – ABroz Mar 17 '21 at 00:05

AWS DocumentDB Syntax differs from MongoDB for UpdateMany

1 Answers1