0

When we insert a document, if the database and/or collection do not exist, they are create before the insert. Is this "create db/collection" operation available in the oplog?

The reason I'm interested, is because we want to upgrade from v2.4 to v3.2 but according to https://jira.mongodb.org/browse/SERVER-17634, the "applyOps" command will fail on insert ops if the collection is missing.

XeniaSis
  • 2,192
  • 5
  • 24
  • 39
  • You will need to upgrade 2.4 to 2.6, then 2.6 to 3.0, then 3.0 to 3.2. It is quite a long chain. If I were you I'd made such upgrade only within a maintenance window without any writes to the db. In this case there is no need to replay opplog. – Alex Blex Oct 16 '17 at 12:09
  • We replay the oplog because we keep mirrors with our clients' data. So whenever the send us data, we have to replay the oplog. However, I don't understand why the update has to be gradual – XeniaSis Oct 16 '17 at 12:17
  • Please read https://docs.mongodb.com/manual/release-notes/3.2-upgrade/index.html regarding upgrades. – Alex Blex Oct 16 '17 at 13:47

1 Answers1

1

Yes, inserting to a new collection makes 2 entries in the oplog:

  1. {op: "c", o: {create: "name_of_the_collection"}} to create the collection

  2. {op: "i", ns: "name_of_the_collection", o:{the document}} to insert the document

Answering the follow up questions:

The official documentation reads:

The oplog (operations log) is a special capped collection that keeps a rolling record of all operations that modify the data stored in your databases.

"all operations" include create collection, database etc. If collection does not exists on primary member, it's created on first insert: https://github.com/mongodb/mongo/blob/v3.2/src/mongo/db/catalog/database.cpp#L454

When new collection created, the corresponding document recorded to the oplog: https://github.com/mongodb/mongo/blob/v3.2/src/mongo/db/catalog/database.cpp#L511

Same logic apply to other commands, including create database.

Format of oplog documents contains all necessary information to replay all operations on correct database, collection, and document:

/* we write to local.oplog.rs:
     { ts : ..., h: ..., v: ..., op: ..., etc }
   ts: an OpTime timestamp
   h: hash
   v: version
   op:
    "i" insert
    "u" update
    "d" delete
    "c" db cmd
    "db" declares presence of a database (ns is set to the db name + '.')
    "n" no op
   bb param:
     if not null, specifies a boolean to pass along to the other side as b: param.
     used for "justOne" or "upsert" flags on 'd', 'u'
*/
Alex Blex
  • 34,704
  • 7
  • 48
  • 75
  • Thanks! Is there any official documentation stating this? I couldn't find anything before posting the question – XeniaSis Oct 16 '17 at 14:30
  • What about the "create db" op? How does `applyOps` know to which db this collection belongs to? – XeniaSis Oct 16 '17 at 15:30
  • I guess it's the same as the `ns`? In the form of `db.collection`? – XeniaSis Oct 16 '17 at 15:42
  • I have updated the answer, but it drifted from the original question. It is recommended to create separate questions, rather than extend existing ones. – Alex Blex Oct 17 '17 at 09:27
  • So basically there is no documentation stating that 2 entries are added in oplog if collection/db doesn't exist. I don't think that the edit to your answer replies any of my follow up questions. It was just a minor confusion of what the `name_of_the_collection` you have in your answer refers to – XeniaSis Oct 17 '17 at 10:06
  • I don't think so. If you check the links in the answer, the logic to create a collection on first insert belongs to database, not replication. For oplog these are 2 different operations, and it does not distinct between implicit or explicit creation of a collection. – Alex Blex Oct 17 '17 at 11:37