1

I have been reading up a lot about CouchDB (and PouchDB) and am still unsure what the best option would be for a project of mine.

I do have a possible way to solve the project in my head based on what I have read so far, but I am unsure about things like performance and would love to get some insights. Or perhaps there's a better place to ask this question? Please let me know if that's the case! (Already tried their IRC channel and the mailing list, but no answers there as of yet)

So the project is basically an 'offline-first' mobile application. The users are device installers. They get assigned a few locations and devices to install every day. They need to walk around buildings and update the data (eg. device X has been installed at location Y; Or property A of device B on location C has been changed to D, etc...)

Some more info about the basic data.

  • There are users, they are the device installers. They need to log into the app.
  • There are locations, all the places that the device installers need to visit.
  • There are devices, all the different devices that can be installed by the users.
  • There are todos, basically a planned installation for a specific user at a specific location for specific devices.

Of course I have tried to simplify the data, but this should contain the gist.

Now, these are important characteristics of the application:

  • Users, locations and devices can be changed by an administrator (back-end software).
  • Todos can be planned by an administrator (back-end software).
  • App user (device installer) only sees his/her own todos/planning for today + 1 week ahead.
  • Multiple app users (device installers) might be assigned to the same location and/or todos, because for a big building there might be multiple installers at work.
  • Automatic synchronization between the data in each app in use and the global database.
  • Secure, it should only be possible for user X to request his/her own todos/planning.

Taking into account these characteristics I currently have the following in mind:

  • One global 'master' database containing all users, locations, devices, todos.
  • Filtered replication/sync using a selector object which for every user replicates only the data that may be accessible for this specific user.
  • Ionic application using PouchDB which does full/normal replication/sync with his/her own user database.

Am I correct in assuming the following?

  • The user of the application using PouchDB will have full read access on his own user database which has been filtered server-side?
  • For updating data I can make use of validate_doc_update to check whether the user may or may not modify something?
  • Any changes done on the PouchDB database will be replicated to the 'user' database?
  • These changes will then also be replicated from the 'user' database to the global 'master' database?
  • Any changes done on the global 'master' database will be replicated to the 'user' database, but only if required (only if there have been new/changed(/deleted) documents for this user)?
  • These changes will then also be replicated from the 'user' database to the PouchDB database for the mobile app?

If all this holds true, then it might be a good fit for this project. At least I think so? (Correct me if I'm wrong!) But I did read some 'performance' problem regarding filtered replication. Suppose there are hundreds of users (device installers) (there aren't this many right now, but there might be in the future). Then would it be a problem to have this filtered replication running for hundreds of 'user' databases? I did read about CouchDB 2.0 and 2.1 having a selector object to do filtered replication instead of the usual JS MapReduce which is supposed to be up to 10x faster. But my question is still: does this work well, even for hundreds (or even thousands) of 'filtered' databases? I don't know enough about the underlying algorithms and limitations but I am wondering whether any change to the global 'master' database does or does not require expensive calculations to run to decide which 'filtered' databases to replicate to. And if it does... does it matter in practice?

Please, any advice would be welcome. I did also consider using other databases. My first approach would actually have been to use a relational database. But one of the required characteristics of this app must be the real-time synchronization. In the past I have been able to handle this myself using revision fields in a RDBMS and with a lot of code, but I would really prefer something as elegant as CouchDB/PouchDB for the synchronization. This is really an area that would save me a lot of headache. Keeping this in mind, what are my options? Am I going in a possible right path or could performance become an issue down the road?

Also note that I have also thought about having separate databases for each user ('one database per user'), but I think it might not be the best fit for this project because some todos might be assigned to multiple users and when one user updates something for a todo, it must be updated for the other user as well.

Hopefully some CouchDB experts can shed some light on my questions. Much appreciated!

I understand there might be some debate but I am only interested in the facts and expertise of others.

Wesley - Synio
  • 654
  • 5
  • 11
  • If you feel the need to plead not to close your question, that's a pretty good sign the question ought to be closed. – Jonathan Hall Aug 27 '17 at 13:13
  • Alright. Edited the last part. I only mentioned this because I saw a related post that was closed for this reason. I wanted to state that I believe my question is more specific and asks whether CouchDB can or cannot be used under the mentioned terms, and under which circumstances it would or would not work well. And if it wouldn't be a good fit, what the possible alternatives could be. – Wesley - Synio Aug 28 '17 at 14:29
  • I can't speak to scaling issues for CouchDB/PouchDB, but you can do this with the Couchbase Mobile stack. Take a look at https://blog.couchbase.com/data-synchronization-with-couchbase-in-ionic-2-hybrid-mobile-apps/ and other posts for information on integrating with ionic. Note I recall there being a tweak needed for ionic 3. You can ping me if you hit a snag. – Hod Sep 02 '17 at 17:20
  • I think you have a good approach to de problem. I've developed an application in a very similar scenario using CouchDB an Cloudant Sync for Android. It has been quite successful with some hundreds of mobile users. As you point out the filtering is a key issue, we try to avoid problems by keeping our databases small developing some custom purging mechanisms. – Juanjo Rodriguez Dec 15 '17 at 21:33
  • May I ask: eventually, what was your decision? Thanks. – Megidd Apr 07 '18 at 13:33
  • 1
    @user3405291 In the end, I used a relational DB for this project and I implemented some custom two-way sync logic through a REST API. – Wesley - Synio Apr 08 '18 at 17:19

0 Answers0