7

MongoDB has a write mechanism of fire and forget. But does this guarantee that my writes will be written to disk eventually? So if i have a statement like this(in pymongo)

collection.insert(doc)     #not passing safe=True

I understand that my write in above statement will not immediately reach the disk but is there a guarantee that it would ever reach the disk(maybe the next day or a week later) or can it get lost and never come back. My application needs dont want the writes to be done synchronously but they want the writes to happen even if its hours later.

lovesh
  • 5,235
  • 9
  • 62
  • 93

3 Answers3

12

The meaning of "fire and forget" (aka the default writes of MongoDB) is that the driver will not confirm the write with the server. Your write is placed on the network, the driver confirms it reaches the network transport but past that no other checking is performed.

As long as the server & network is up, it is expected the write will succeed. However, this also means you won't find out about issues such as violating a unique index.

Writes initially occur in memory, and are flushed to disk at least every 60 seconds. If you have journalling turned on, the write will get into the journal within 100 ms – this journal can recover in case of a crash upon mongod restarting.

If you want to verify that a write has made it to mongod and successfully been applied in memory, you should set safe=True

Brendan W. McAdams
  • 9,379
  • 3
  • 41
  • 31
  • So does that mean if i have a unique index i should always use safe while writing a document that has the unique field – lovesh Jul 20 '12 at 05:03
  • 3
    If you want to verify that the document was actually inserted and not dropped because of a duplicate key error, then yes, you need to call getLastError on the connection (which is what a safe=true write will do). If you knew beyond all doubt that the uniqueness constraint was never going to be violated, or if you wished for any dupes to be dropped by design then you would not have to use a safe write. – Adam Comerford Jul 21 '12 at 01:08
  • `dropDups` is weird. when it is enabled and i try to insert a single document that violates the unique constraint it throws a DuplicateKey Exception but when i try to insert a bulk of documents then it silently drops the documents which dont satisfy the unique constraint – lovesh Jul 21 '12 at 09:23
1

AFAIK Fire and Forget has no guarantees - if you need them then use safe [that's what it is there for!]

See How safe is MongoDB's safe mode on inserts?

Community
  • 1
  • 1
NPSF3000
  • 2,421
  • 15
  • 20
  • AFAIKnew safe was there for guaranteeing that your write has successfully made to the database then and there. So its used for synchronous writes. I though not using `safe` would make my writes asynchronous but they would eventually make it to the database. But now i have realized that its not true. `The write might just get lost` – lovesh Jul 20 '12 at 04:52
0

It can get lost and never come back. This can happen due to a power outage, hardware failure, or other environmental problem. It can also happen if the doc violates some unique index.

safe=True will ensure that there weren't any database engine errors doing the insert. This makes the driver issue a getLastError command immediately after the write, and it will return an error if the write violates a unique index or there was some underlying Mongo problem. This won't help you if you lose power between the in memory write and the write to the journal, though.

There are numerous other options for getLastError that you can pass through the safe option to ensure that a write either makes it to the journal (durable), or is replicated before the command returns. We generally tell people to use the journaling option at a minimum for writes they care about.

MrKurt
  • 5,080
  • 1
  • 23
  • 22
  • 1
    I don't think the power outage and other full instance failure scenarios are covered by safe=true (w=1) writes necessarily. Safe=true writes do very little more than report back on any logical write failures (index constraints, syntax issues, OOM issues, etc.). – Remon van Vliet Jul 19 '12 at 16:30
  • I'll clarify! There wasn't much of a transition between sentences, eh? – MrKurt Jul 19 '12 at 21:39
  • I think SO is also using MongoDB without doing safe inserts. I just upvoted your answer and i can see the up arrow change to orange without getting the count incremented. – lovesh Jul 20 '12 at 04:52