0

I have millions of documents in various collection.

My application has to periodically update some incremental data to MongoDB. How do I acheive this?

For every record in incremental data, if the record exists it must be updated with new values otherwise it has to be inserted. 

{
   _id: 'some ID',
   lifnr: 12345,
   bukrs: 3455,
   amount: 500
},
{
   _id: 'some ID',
   lifnr: 12346,
   bukrs: 3456,
   amount: 5200
}

Assuming above is my data, with lifnr and bukrs as compound index with unique constraint. Any new records with {lifnr & bukrs} value same as any of existing should replace the old amount. If it is a new record of unique {lifnr & bukrs}, then it must insert/append it.

Guide me to the approach. Thanks

Kavitha Madhavaraj
  • 562
  • 1
  • 6
  • 23
  • More info on exactly what data and queries (operators being used as well) would help – Sammaye Dec 24 '15 at 10:44
  • 1
    Am I over simplifying this or would something like: `db.col.update({lifnr:doc.lifnr,bukrs:doc.bukrs},doc,{upset:true})` be sufficient? That will update with new values or insert – Sammaye Dec 24 '15 at 10:55
  • @Sammaye Could please explain it? I'm new to Mongo ! By the way isn't it 'upsert'? Thanks – Kavitha Madhavaraj Dec 24 '15 at 10:58
  • basically `doc` is a var that contains your changed document, you query for an update by the two conditional values in your compound index (since exception throwing does not work in query) and then you replace the old doc with the new in the second param or you use upset:true to insert a new doc if not found – Sammaye Dec 24 '15 at 10:59

1 Answers1

1

Based on your edit, its not really incrementing data. Its just an update with upsert set to true. Do update every time. The query ill be,

db.collection.update(
  { lifnr: 12345, bukrs: 3455},
  {$set: {amount: 5600}}
  {upsert: true}
)

The above query would update amount, if the linfr & bukrs combination is found. If not found it will insert a new document.

Sarath Nair
  • 2,828
  • 2
  • 21
  • 36
  • I had clearly mentioned as 'incremental data' and not as 'incrementing data'. Thanks for your answer though ! – Kavitha Madhavaraj Dec 24 '15 at 11:16
  • So after trying your solution for millions of records, I experienced a great delay.Hence, I switched to bulk upsert. It updates but not inserts. Code --> bulk.find(check_fields).upsert().update({"$setOnInsert": entire_doc, "$set":document }); – Kavitha Madhavaraj Jan 13 '16 at 09:48
  • Bulk update is a very good alternative for millions of records – Sarath Nair Jan 13 '16 at 10:23