4

After doing some research, it seems that when people ask about schema migration in CouchDB, they get quickly rebutted that the concept is not exactly compatible with NoSQL ideology. I understand how NoSQL is meant to be schema-less and it feels counter-intuitive. Nevertheless, I am in such a situation that I do not see how schema migration can be totally avoided. I use NoSQL with mobile devices. Here's the big lines of my set-up and my needs.

  1. Each mobile device has its own replicated database.
  2. The mobile device can work offline.
  3. Mobile devices replicates with a database on the cloud server to share their data.
  4. The data replicates when possible so multiple users on different devices can work on the same data.
  5. A newer version of the client could decide to change the value type it stores for a particular key (i.e. the equivalent of a schema change)
  6. The users might not update the app as soon as an update is available.
  7. A user that is not updated should be able to still work on the same data as the users who are updated.

Edit:

  1. The application is 100% native code.

My problem here is that a user with a client that is an old version of the software would not expect the new data to be a string instead of an int per se. Thus, handling it client-side can not always be done since we do not want to force the users to update their apps.

To explain it under another angle what happens is that I want two users with two different client versions to be able to share and modify the same data, meaning that I need to have backward and forward compatibility. Backward compatibility is often tackled, but forward compatibility is a bit more problematic. I need a way to continue to use my last-modified heuristic for conflict resolution while having a version that can not interpret the new schema fully.

After some thinking, I could consider an approach where I should split the cloud database into multiple sources; one where every mobile device pushes its modifications and one-to-x databases where the mobile devices pull the most recent data adapted to their versions. Somewhere in between those I should handle backward and forward compatibility with schema migrations.

Is there a way to avoid schema migrations, even in such a set-up? Do anyone see any other simpler or safer solutions? What are the limitations of such a system? I do not feel that I am exactly on the right track here.

Thanks, Paul

airpaulg
  • 565
  • 3
  • 13
  • 1
    I'd caution you from taking too much stock in rebuttals around schema migration and NoSQL databases. "They" may say there aren't schemas, but if there's any design document, or even code that expects the structure to be in a certain way, to me that's a schema (documented or not). Regardless of a formal schema, changing the structure is a real problem. It seems like you have a challenge if older clients can access/manage/save newer data structures. Seems like forcing users to update if a given document has been already upgraded to a new version might be the safest/most reliable. – WiredPrairie Apr 25 '13 at 15:52
  • Thank you for the answer. Forcing users to update is a possible solution, but it does impose a constraint on the users that I do not want. As of now, I am trying to focus on a solution that would have the least impact on the user experience. – airpaulg Apr 25 '13 at 17:09

1 Answers1

2

If CouchApps are an option for you, then you could actually circumvent the problem of different client versions. Just in case you are not familiar with these: CouchApps are HTML+CSS+JavaScript based apps which are served directly out of CouchDB. On a mobile platform it might come in handy to embed them in Cordova (Phonegap) for access to native functions.

As CouchApps are essentially stored in design documents in a CouchDB database, they can be replicated in the same way as ordinary documents can be replicated. That gives you the option to perform an update of the client software whenever the client is connected to the server (actually it would happen automatically on any replication).

In case you want to spend a thought on this option, you might want to have a look at garden20, Mobile Futon, kanso and erica.

brdlph
  • 619
  • 4
  • 15
  • My apologies, I forgot to mention that my application is 100% native. I do not think it would be a viable option in those conditions. Even if it was, I still wonder what you have in mind. Would CouchApps be in charge of doing forward compatibility operations every time an old version receives new data via pull? Thank you for your answer! – airpaulg Apr 25 '13 at 17:19
  • Well, then this is unfortunately no option for you. But just to clarify: The "forward compatibility operation" would be a simple replacement of the outdated design document by a new one. This would happen automatically on any replication with the server given that there is a newer version of the design document available. The core idea is that application code and data are handled in the same way by replication. – brdlph Apr 26 '13 at 07:33