1

A client of ours is creating hourly mongo dumps and sends them over to us. We, then, replay the oplog. However, the client decided that he doesn't want to send over all dumps anymore, but only those needed for debugging in emergency cases. However, my worry is, that not sending everything will lead to inconsistencies on our local mirror.

My main question is the following: what happens when you replay the oplog that contains an update op for a document that's missing? Will it error out or just ignore the operation?

To explain better what I mean, here is an example case:

  • client creates entry with {_id: 1} on 2017-11-27
  • same entry is first updated on 2017-11-28
  • client sends over oplog containing only operations from 2017-11-28 forward
  • we replay oplog locally

What will happen to the update op for entry with {_id: 1}? Will it error out? Will it be ignored since there is not document to be updated with such _id?

P.S. currently using mongo 2.4

XeniaSis
  • 2,192
  • 5
  • 24
  • 39

2 Answers2

1

However, my worry is, that not sending everything will lead to inconsistencies on our local mirror.

This is correct. Not sending everything means that your copy is not a mirror anymore, but more like a fork. In other words, it's likely to be useless for troubleshooting/reproduction purposes.

The oplog applier assumes that the database is in a certain state for it to work. Applying the oplog to a random state makes little sense, and it was never designed to work like this. Even if it doesn't crash with an assertion error, the database is in an undefined state.

You can read more about MongoDB replication internals in https://github.com/mongodb/mongo/wiki/Replication-Internals

P.S. currently using mongo 2.4

Note: MongoDB 2.4 is very old and is not supported anymore. The latest MongoDB version as of this writing is 3.4.10. Please consider upgrading to a supported release.

kevinadi
  • 13,365
  • 3
  • 33
  • 49
  • We really can't do anything about the inconsistency of the db. However, my actual question was if applying the oplog is going to work or crash due to missing documents. Couldn't find any documentation on that. It's in the works to update mongodb but cannot do it immediately – XeniaSis Nov 28 '17 at 08:31
  • @XeniaSis it will crash – kevinadi Nov 28 '17 at 21:32
  • @XeniaSis to clarify, some oplog operation will crash the database such as updating a document in a dropped collection. The oplog idempotency could probably cope with some cases of missing documents, but there is no guarantee. – kevinadi Nov 28 '17 at 22:11
0

If customer has sent ALL oplogs to you after last dump, then there is no missing information what would prevent oplog apply, because all information (insert, update, delete) is in the oplog.

And yes. Version 2.4 is very old...

JJussi
  • 1,540
  • 12
  • 12