bulk_write not working in mongo db, how to write new documents to collection?

Question

python : 3.8.8 pymongo : 3.12.9

I have data in my collection and new data that I want to push it to the collection.

The data in my database

from pymongo import MongoClient, UpdateOne
import pandas as pd

{
    "_id" : ObjectId("6143b5b78d248ba8ed0c3d88"),
    "Sector" : "A",
    "RKK_T" : 0.15,
    "RKK_M" : 0.2,
    "lastModified" : ISODate("2021-09-16T22:23:45.411Z")
},
{
    "_id" : ObjectId("6143b5b78d248ba8ed0c3d89"),
    "Sector" : "B",
    "RKK_T" : 0.22,
    "RKK_M" : 0.3,
    "lastModified" : ISODate("2021-09-16T22:23:45.411Z")
}

And this is my new data that I want to push it to the database

data = [{
    "_id" : "6143b5b78d248ba8ed0c3d88",
    "Sector" : "A",
    "RKK_T" : 0.15,
    "RKK_M" : 0.25,
    "lastModified" :datetime.utcnow()
},
{
    "_id" : "6143b5b78d248ba8ed0c3d89",
    "Sector" : "B",
    "RKK_T" : 0.22,
    "RKK_M" : 0.3,
    "lastModified" : datetime.utcnow()
},
{
    "_id" : "6143b5b78d248ba8ed0c3d90",
    "Sector" : "C",
    "RKK_T" : 0.25,
    "RKK_M" : 0.32,
    'RKK_K' : 0.4,
    "lastModified" : datetime.utcnow()
}
]

    df = pd.DataFrame(data, columns=['_id', 'Sector', 'RKK_T', 'RKK_M', 'lastmodified'])
    df

    _id                     Sector  RKK_T   RKK_M   lastmodified
0   6143b5b78d248ba8ed0c3d88    A   0.15    0.25    NaN
1   6143b5b78d248ba8ed0c3d89    B   0.22    0.30    NaN
2   6143b5b78d248ba8ed0c3d90    C   0.25    0.32    NaN

So I think I need to filter by each Sector and update values of RKK_T and RKK_M columns.

based on the solution here I tried

a = []
b = {}
for d in data :
    b = {'updateOne':{
          "filter": {'Sector':d['Sector']},
           "update":{'$set':{
                      "RKK_T":d["RKK_T"],
                       "RKK_M" : d["RKK_M"],
                       'RKK_K' :d["RKK_K"],
                       "lastModified" :d["lastModified"]
                     }}
         }}
    a.append(b)
    db.collection.bulk_write(a)

but got this error

AttributeError: 'dict' object has no attribute '_add_to_bulk'

score 0 · Accepted Answer · answered Sep 20 '21 at 20:02

0

The construct for pymongo's UpdateOne() bulk operator is documented here: https://pymongo.readthedocs.io/en/stable/api/pymongo/operations.html#pymongo.operations.UpdateOne

An example of use:

from pymongo import UpdateOne

# <setup snipped>

for d in data:
    b = UpdateOne({'Sector': d['Sector']}, {'$set': {
        "RKK_T": d.get("RKK_T"),
        "RKK_M": d.get("RKK_M"),
        'RKK_K': d.get("RKK_K"),
        "lastModified": d.get("lastModified")
    }})
    a.append(b)

db.collection.bulk_write(a)

answered Sep 20 '21 at 20:02

Belly Buster

8,224
2
7
20

should we add `upsert` to make sure adding the records that do not exist in the db ? – Alexander Sep 20 '21 at 20:53
yes that would work. – Belly Buster Sep 20 '21 at 23:15
Great! How could I reach you if I have further questions on pymongo and mongodb related? If I mention you with @Belly Buster in the question would you see it ? – Alexander Sep 20 '21 at 23:17
1

Raise a ticket and I might see it. I keep an eye on pymongo Qs but many others do too. – Belly Buster Sep 21 '21 at 07:56

bulk_write not working in mongo db, how to write new documents to collection?

1 Answers1