Optimistic Locking in BASE style databases

Question

BASE style databases are Soft State and Eventually Consistent. I'm aware that different database management systems vary and their configurations make a huge difference. But let's imagine this:

Let's say I have a NoSQL cluster with 2 nodes. It's eventually consistent. When I write to Node1, it's replicated to Node2 after some time. Let me implement an Optimistic Locking mechanism by comparing the versions. So, when I try to write to Node1, write operation will not be successful if my version is behind the version in the database. But when there are two threads (or process) try to update the same row, something like this can happen:

1. thread1 reads the data with version=0 from Node1
2. thread2 reads the data with version=0 from Node2
3. thread1 updates the data with version=1 on Node1
4. thread2 tries to update the data with version=1 on Node2 and it may success because write operation may not be replicated yet.

Am I missing something? How do some BASE style database systems solve this? Can you give me solid examples about which database system solves this problem how?

Answer 1

I'm not sure this is the best forum for this kind of question, because it might involve discussion and back-and-forth. So you may want to check out the Couchbase Discord.

However, I can tell you that for Couchbase (which might meet the "BASE" criteria as you have described it, that's a whole other conversation), the method by which locking is implemented is called "Compare-and-Swap" (aka CAS). From the documentation:

CAS, or Compare And Swap, is a form of optimistic locking. Every document in Couchbase has a CAS value, and it’s changed on every mutation. When you get a document you also get the document's CAS, and then when it’s time to write the document, you send the same CAS back. If another thread or program has modified that document in the meantime, the Couchbase Server can detect you've provided a now-outdated CAS, and return an error. This provides cheap, safe concurrency.

And there's more detail on how this works at an SDK level in the documentation (for instance, the Java SDK Concurrent Document Mutations docs).

As a end user of the database, all you need to worry about is holding on to the CAS value, as it's the key to the optimistic lock (and pessimistic lock, which is also an option in Couchbase). Comparing the CAS values for optimistic locking will often require a retry loop just in case the document is under contention. (And if it's often under heavy contention, consider a pessimistic lock instead).

If you want to get deeper into the actual CAS implementation (i.e. you're writing your own database, I guess), I would again point to the Couchbase Discord, as the core engineers often hang out there.

Answer 2

Matthew Groves' answer is spot on. I'll just add a bit more context.

With Couchbase, Key/Value writes are immediately visible to everyone.

That's because each document is assigned to a partition based on its document ID. There's an "active" version of the partition, as well as "replicas" distributed across the cluster. A Couchbase client SDK only talks to the active partition, unless you explicitly tell it to read from a replica.

1. thread1 reads the data with version=0 from Node1
2. thread2 reads the data with version=0 from Node2

This means that in your example, thread1 and thread2 will talk to the same node (as long as they're accessing the same document). The exception is that during a cluster rebalance, replicas might be promoted to "active". That's fine for optimistic locking, though, since the replica document has the same Compare-And-Swap value as the document from the active partition (as long as it's really the same version, and not stale).

See Understanding vBuckets for more info.

And also this post from the Couchbase forum: https://www.couchbase.com/forums/t/how-can-i-archive-strong-consistency/17530