Mongorestore to update records if already exists without --drop

Question

I have a huge dump of collection of data which i have to transfer to another machine every weekend. So i'm planning for an incremental backup and restore. As experimented, mongorestore never merges data if _id already exists. Based on the above problem i tried using mongoimport and export with but the same problem exists as the existing records are not merged. Any possible solution would be helpfull.

error in mongoimport command caused by :: 11000 E11000 duplicate key error index: news.news_data.$id dup key: { : ObjectId('5404410d9f5323ef734dac68') }

Exact duplicate of [mongorestore command replace existing records?](https://stackoverflow.com/questions/27666908/mongorestore-command-replace-existing-records) — Dan Dascalescu, Jun 10 '19 at 17:41

score 20 · Accepted Answer · edited May 28 '21 at 04:02

20

The first case is true. Mongorestore does not update documents if already exist. It skips those documents when restoring. In your second case, please try using mongoimport with --upsert option. It will merge the records if _id already exists.

Example:

mongoimport --db dbname --collection collname --upsert --file file.json

edited May 28 '21 at 04:02

Pang

9,564
146
81
122

answered Sep 22 '14 at 06:34

Kumar Kailash

1,385
1
10
19

1

Be warned that mongoimport imports json, which doesn't have all the type information that mongorestore, which uses bson, will have. Also, I don't quite see what the problem is- if you're transferring a dump, why not drop the old data so there's no duplicates? – wdberkeley Sep 22 '14 at 18:31
3

@wdberkeley: some potential reasons for preferring an [upsert](https://jira.mongodb.org/browse/TOOLS-121) to a drop + restore: only a few documents are updated in a large collection; not wanting to waste time recreating indexes; emitting only the necessary changes for changeStreams. – Dan Dascalescu Jun 10 '19 at 17:38

score 6 · Answer 2 · edited Jun 10 '19 at 17:40

6

Please vote for this ticket to add an upsert option to mongorestore.

Until that is implemented, we have found a workaround:

Dump Collection From Server A ( original )
Dump Collection From Server B ( target )
Restore With Drop Collection A on Server B 4
Restore Without Drop Collection B on Server B

In this case updated document will not overwritten

edited Jun 10 '19 at 17:40

Dan Dascalescu

143,271
52
317
404

answered May 11 '16 at 09:17

Francesco Grotta

61
1
1

Mongorestore to update records if already exists without --drop

2 Answers2