1

I have embedded sub documents array in MongoDB document and multiple users can try add sub documents into the array. I use update($push) query to add document into array, but if multiple users try to add entry from UI, how do I make sure second $push doesn't fail due to lock by first? I would have chance of only couple of users adding entry at same into single document, so not to worry about the case where 100s of users exist. What is the default wait time of update in WiredTiger, so 2nd push doesn't abort immediately and can take upto 1 sec, but $push should complete successful?

I tried finding default wait time in MongoDB and WiredTiger docs, I could find transaction default wait times, but update query.

Nar
  • 75
  • 1
  • 6

1 Answers1

2

Internally, WiredTiger uses an optimistic locking method. This means that when two threads are trying to update the same document, one of them will succeed, and the other will back off. This will manifest as a "write conflict" (see metrics.operation.writeConflicts).

This conflict will be retried transparently, so from a client's point of view, the write will just take longer than usual.

The backing off algorithm will wait longer the more conflict it encounters, from 1 ms, and capped at 100 ms per wait. So the more conflict it encounters, it will eventually wait at 100 ms per retry.

Having said that, the design of updating a single document from multiple sources will have trouble scaling in the future due to two reasons:

  1. MongoDB has a limit of 16 MB document size, so the array cannot be updated indefinitely.
  2. Locking issues, as you have readily identified.

For #2, in a pathological case, a write can encounter conflict after conflict, waiting 100 ms between waits. There is no cap limit on the number of waits, so it can potentially wait for minutes. This is a sign that the workload is bottlenecked on a single document, and the app essentially operates on a single-thread model.

Typically the solution is to not create artificial bottlenecks, but to spread out the work across many different documents, perhaps in a separate collection. This way, concurrency can be maintained.

kevinadi
  • 13,365
  • 3
  • 33
  • 49
  • Thank You @kevinadi. Do you know if MongoDB has this information in it's docs or tutorials? "The backing off algorithm will wait longer the more conflict it encounters, from 1 ms, and capped at 100 ms per wait. So the more conflict it encounters, it will eventually wait at 100 ms per retry." – Nar Jul 06 '19 at 15:08
  • No this is not documented since it's an internal setting that may change from time to time. From end-user's point of view, only the write conflict metric matters. For today, this is how it's done. – kevinadi Jul 07 '19 at 21:58