I was unable to find proper solution to handle "Eventual conflicts" (see http://pouchdb.com/guides/conflicts.html for "Eventual conflicts" description) when I have document with multiple unique field constraints, like unique username constraint and unique email constraint for user registration document or Article document with unique name and title + all the documents are replicating live on mobile devices using Ionic + PouchDB and Couchbase Sync Gateway.
Let's take for example Article document with unique name constraint and unique title constraint.
1). I have Article document which looks like this:
{
"_id": "article::3808cea0-bb99-11e5-9ba4-315d40280ab9",
"_rev": "1-8b8076d3fef9994ca65a516532c5f975"
"type": "Article",
"name": "name1",
"title": "title1",
"owner": "user-id-here"
}
2). Also, I'm creating separate constraint documents for each unique field (see Unique constraints in couchdb and https://kfalck.net/2009/06/29/enforcing-unique-usernames-on-couchdb) with "_id" key in format "{doc type}--constraint-unique--{unique property / field name}::{hashed value of the property in sha256}".
So, for the above doc with "name1" and "title1" I'm creating two "ConstraintUnique" docs / placeholders for name field and for title field.
Unique "name" field placeholder doc:
{
"_id": "article--constraint-unique--name::e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
"_rev": "1-de7e0636632764e97880f362d50598b7",
"type": "ConstraintUnique",
"docRef": "article::3808cea0-bb99-11e5-9ba4-315d40280ab9",
"propertyName": "name",
"propertyValue": "name1"
}
Unique "title" field placeholder doc:
{
"_id": "article--constraint-unique--title::5018e26faaec56a465e61aaeb752c2984c8a6d6305c8496b3a95213fb20889c7",
"_rev": "1-34c1771f2108f6ad8efc0d99e79d78a0",
"type": "ConstraintUnique",
"docRef": "article::3808cea0-bb99-11e5-9ba4-315d40280ab9",
"propertyName": "title",
"propertyValue": "title1"
}
On all "ConstraintUnique" docs I have "docRef" field which points to "_id" of actual doc like "Article._id"
3). This way, if I want to add in the future more unique constraints I will just add another "ConstraintUnique" and when I'm saving/creating a new article I'm checking for existing "constraint docs" and then save or reject the original doc. Another option will be to insert the doc without additional "constraint docs" and check if there is a duplicate using views and delete the duplicate straight away. (all is described here: Unique constraints in couchdb).
The problem:
I'm doing live bidirectional replication (client to server and server to client sync) like this:
db.sync('remote db url', {
live: true,
retry: true
})
Then, I listen for changes like this:
db.changes({
since: 'now',
live: true,
include_docs: true,
conflicts: true
}).on('change', function(change) {
if (change.doc._conflicts) {
// do coflict resolution
}
});
and I have following scenario where both users are OFFLINE and NO data is synced to remote DB, but it is inserted in their local dbs:
1). User1 is offline and he inserted Article with name1 and title1, his local db will look like this:
{
"_id": "article--constraint-unique--name::e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
"_rev": "1-de7e0636632764e97880f362d50598b7",
"type": "ConstraintUnique",
"docRef": "article::3808cea0-bb99-11e5-9ba4-315d40280ab9",
"propertyName": "name",
"propertyValue": "name1"
}
{
"_id": "article--constraint-unique--title::5018e26faaec56a465e61aaeb752c2984c8a6d6305c8496b3a95213fb20889c7",
"_rev": "1-34c1771f2108f6ad8efc0d99e79d78a0",
"type": "ConstraintUnique",
"docRef": "article::3808cea0-bb99-11e5-9ba4-315d40280ab9",
"propertyName": "title",
"propertyValue": "title1"
}
{
"_id": "article::3808cea0-bb99-11e5-9ba4-315d40280ab9",
"_rev": "1-8b8076d3fef9994ca65a516532c5f975"
"type": "Article",
"name": "name1",
"title": "title1",
"owner": "user1"
}
2). User2 is also offline, and he is inserting same Article with name1 and title1, so his local db will look almost the same, but with different Article._id and the same ConstraintUnique ids:
Same as User1's (except _rev and docRef):
{
"_id": "article--constraint-unique--name::e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
"_rev": "1-5c427cba5efe71114acb7e6ddd8b8a23",
"type": "ConstraintUnique",
"docRef": "article::ad16f2e4-9cff-42ae-ae92-27e353e7f229",
"propertyName": "name",
"propertyValue": "name1"
}
Same as User1's (except _rev and docRef):
{
"_id": "article--constraint-unique--title::5018e26faaec56a465e61aaeb752c2984c8a6d6305c8496b3a95213fb20889c7",
"_rev": "1-14244246dbbac7e55e132abe47fe115d",
"type": "ConstraintUnique",
"docRef": "article::ad16f2e4-9cff-42ae-ae92-27e353e7f229",
"propertyName": "title",
"propertyValue": "title1"
}
Different, but the actual data is the same:
{
"_id": "article::ad16f2e4-9cff-42ae-ae92-27e353e7f229",
"_rev": "1-8b8076d3fef9994ca65a516532c5f975"
"type": "Article",
"name": "name1",
"title": "title1",
"owner": "user2"
}
3). The actual problem:
When User 1 is back online and his changes are synced with the remote database without conflict:
Ids of the docs:
"_id": "article--constraint-unique--name::e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"
"_id": "article--constraint-unique--title::5018e26faaec56a465e61aaeb752c2984c8a6d6305c8496b3a95213fb20889c7"
"_id": "article::3808cea0-bb99-11e5-9ba4-315d40280ab9"
Later, User 2 is also back online and his changes are synced with the remote database WITH conflict:
Ids of the docs:
"_id": "article--constraint-unique--name::e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" -- conflicting with the User1
"_id": "article--constraint-unique--title::5018e26faaec56a465e61aaeb752c2984c8a6d6305c8496b3a95213fb20889c7" -- conflicting with the User1
"_id": "article::ad16f2e4-9cff-42ae-ae92-27e353e7f229" -- no conflict, but I need to remove this duplicate somehow
So, CouchDB / Couchbase Sync Gateway chooses the winner for "article--constraint-unique--name::e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" and "article--constraint-unique--title::5018e26faaec56a465e61aaeb752c2984c8a6d6305c8496b3a95213fb20889c7" on the server side, but PouchDB on the client also syncs the duplicated Article doc - I want to clean up constraint docs conflicts ("article--constraint-unique--name::e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" and "article--constraint-unique--title::5018e26faaec56a465e61aaeb752c2984c8a6d6305c8496b3a95213fb20889c7") and update docRef (if the docRef is not the same on both "name" and "title" constraints), also I want to remove the duplicated Article with ids "article::3808cea0-bb99-11e5-9ba4-315d40280ab9"(User1) and "article::ad16f2e4-9cff-42ae-ae92-27e353e7f229"(User2) or better I want to choose one winning Article, then update docRef in "article--constraint-unique" docs. Any solution is welcome - I just want to remove duplicated Article and update constraint placeholders to be in sync with the winning Article.
I've done raw implementation where I check for conflicts in changes feed (http://pouchdb.com/guides/changes.html), I am using Ionic 1/AngularJS 1 + PouchDB and try to delete duplicated doc/Article:
var changeListener = db.changes({
since: 'now',
live: true,
include_docs: true,
conflicts: true
}).on('change', function(change) {
var doc = angular.copy(change.doc);
var i;
if (doc._conflicts) {
if (doc.type && (doc.type === 'ConstraintUnique')) {
for (i = 0; i < doc._conflicts.length; i++) {
(function(currentConflictRev) {
// get conflicted "ConstraintUnique"
db.get(doc._id, {rev: currentConflictRev}).then(function(conflictConstraintUniqueDoc) {
if (conflictConstraintUniqueDoc.docRef) {
// get linked to "ConstraintUnique" doc and delete it
db.get(conflictConstraintUniqueDoc.docRef)
.then(function(conflictDoc) {
// delete the duplicated doc
db.remove(conflictDoc._id, conflictDoc._rev);
// resolve the current conflict
db.remove(conflictConstraintUniqueDoc._id, conflictConstraintUniqueDoc._rev);
});
}
});
})(doc._conflicts[i]);
}
}
}
});
This implementation is not working when Couch server decides that winning "ConstraintUnique" doc for name is with docRef==user1Article and the other winning "ConstraintUnique" doc for title is with docRef==user2Article - this way both articles are deleted, not only the duplicated one.
I'm sure, I'm doing it wrong: so, how to deal with conflicts and offline client sync where we use documents with unique constraints on their fields in case of eventual conflict / master-master conflict?
More topics are read with no luck on solution:
Thank you very much in advance!
