Performing Data Cleanup In Mongodb

Question

My application tracks the movements of data throughout the system. When a movement is recorded it is placed in a separate collection that determines whether the document is enroute, available or out of service. I used $addToSet to place the _id, and $pullAll to make sure that when a doc is moved from enroute to available, it is not duplicated. But when the _id is moved to a new location entirely, I need to remove the old data from the old location and insert it into the new location. The insertion works but I cannot figure out how to properly remove the data from the old location. These are all down within Meteor Calls and Mongodb

          if last.status is "Enroute"
            LastLocation.update locationId: last.locationId,partId: last.partId,
                $addToSet:
                    enroutePurchaseIds: lastPurchaseId
                $pullAll:
                    availiblePurchaseIds: lastPurchaseId
                    outOfServicePurchaseIds: lastPurchaseId

` The insertion works but I cannot figure out how to properly remove the data from the old location` does this refer to the `$pullAll` operation? Can you add some short set of sample data, the current result and the expected result? — Jankapunkt, Mar 26 '20 at 12:58
Since, you are using the `$pull` and `$addToSet`, this means you are maintaining these data _id's in arrays (within documents). You know the `_id` value of the tracked document, and the _old location_ (this must be a field within the document where the dat is tracked). So, you can query that document and update it ($pull is an update operation). _"... but I cannot figure out how to properly remove the data from the old location."_ can you share this issue in little more detail? — prasad_, Mar 31 '20 at 11:21

s7vr · Answer 1 · 2020-04-01T00:15:57.383

Update

You can run the merge command from upcoming 4.4 version which allows updating the same collection the aggregation is running on. Pass the array as old location and new location

db.collection.aggregate([
  {"$match":{"location":{"$in":[oldLocation,newLocation]}}},
  {"$addFields":{"sortOrder":{"$indexOfArray":[[oldLocation,newLocation],"$location"]}}},
  {"$sort":{"sortOrder":1}},
  {"$group":{
    "_id":null,
    "oldLocationDoc":{"$first":"$$ROOT"},
    "newLocationDoc":{"$last":"$$ROOT"}
  }},
  {"$addFields":{
    "oldLocationDoc.old":{
      "$filter":{
        "input":"$oldLocationDoc.old",
        "cond":{"$ne":["$$this",oldLocation]}
      }
    },
    "newLocationDoc.new":{"$concatArrays":["$newLocationDoc.new",[newLocation]]}
  }},
  {"$project":{"locations":["$oldLocationDoc","$newLocationDoc"]}},
  {"$unwind":"$locations"},
  {"$replaceRoot":{"newRoot":"$locations"}},
  {"$merge":{
    "into":{"db":"db","coll":"collection"},
    "on":"_id",
    "whenMatched":"merge",
    "whenNotMatched":"failed"
  }}
]

Original

Not possible to move array/field value from one document to another document in a single update operation.

You would want to use transactions to perform multi document updates in a atomic way. Requires replica set.

var session = db.getMongo().startSession();
var collection = session.getDatabase('test').getCollection('collection');
session.startTransaction({readConcern: {level:'snapshot'},writeConcern: {w:'majority'}});
collection.update({location:oldLocation},{$pull:{availiblePurchaseIds:lastPurchaseId}});
collection.update({location:newLocation},{$push:{enroutePurchaseIds:lastPurchaseId}});
session.commitTransaction()
session.endSession()

Other options would be to perform bulk updates in case of standalone mongod instance.

var bulk = db.getCollection('collection').initializeUnorderedBulkOp();
bulk.find({location:oldLocation}).updateOne({$pull:{availiblePurchaseIds:lastPurchaseId}});
bulk.find({location:newLocation}).updateOne({$push:{enroutePurchaseIds:lastPurchaseId}});  
bulk.execute();

score 0 · Answer 2 · answered Mar 12 '20 at 18:44

0

Are you moving the entire document from one collection to another or just moving the document's id? I can't help much with coffeescript but if you're looking to move entire documents you might find the following thread helpful.

mongodb move documents from one collection to another collection

answered Mar 12 '20 at 18:44

thatsmycode

21
2

Thanks for the article but no I am not moving one document from on collection to another. I am moving Id's from one document to another. When the id is placed into another document I need it removed from the document it was coming from. – Geoffrey Hutson Mar 13 '20 at 16:58

Performing Data Cleanup In Mongodb

2 Answers2