Perform UPSERT with update_one() using PyMongo while also adding/setting a new field with every document

Question

I have a python request returning api data in json format like below:

[
 {
  'name': 'xyz',
  'address': 'yyy'
 },
 {
  'name': 'abc',
  'address': 'ccc'
 }
]

I want to load this data in MongoDB every day as such irrespective of if there have been any changes to address from the api json response output or not, along with a datetimestamp field added with every document insert/update for each name key in json object, like below:

[
     { 
      'id' : objectid(123456),
      'name': 'xyz',
      'address': 'yyy'
      'dttm' : '2020-08-26:01:20:30'
     },
     {
      'id' : objectid(123457),
      'name': 'abc',
      'address': 'ccc'
      'dttm' : '2020-08-26:01:20:30'
     }
     { 
      'id' : objectid(123458),
      'name': 'xyz',
      'address': 'zzzzzzzzzzzzzzzzzzz'
      'dttm' : '2020-09-26:03:01:20'
     },
     {
      'id' : objectid(123459),
      'name': 'abc',
      'address': 'ccc'
      'dttm' : '2020-09-26:03:02:30'
     }
 ]

In above example - name "xyz" has address change from previous run, but name "abc" had no change but still loading with a different datetimestamp

Sample code:

import requests
import json
from pymongo import MongoClient

url='xxx'
header={'Auth':'xyz', 'Content-Type': 'application/json'}
payload={}
api_data = requests.request("POST", url, data=json.dumps(payload), headers=header).json()

client = MongoClient('mongodb://localhost:27017') 
db = client[test_db][test_collection]

db.update_one({api_data}, {$set:{dttm:datetime.today()}},{'upsert':True}) ##===>>what is the valid way to construct this using PyMongo ??

client.close()

Answer 1

You can provide the timestamp in the update document, it will be written into the database on both inserts and updates.

For upsert, see Fast or Bulk Upsert in pymongo.