1

My NodeJS project is based on SailsJS, itself using ExpressJS.

Its API will be used by mobile apps to fetch their data from it.

The tricky part is I don't want the client apps to fetch the whole data tree every time there is a change in the database.

The client only needs to download a differential between the data it's already got and the data on the server.

To achieve that I thought of using git on the server. That is create a repository and save all endpoints as a json file in the repo. Each save will trigger an automatic commit.

Then I could create a specific API endpoint that will accept a commit sha as a parameter and return a diff between that and git HEAD.

This post by William Benton comforted me with this idea.

I'm now looking for any tips that could help me get this working based on the language and frameworks cited above :

  • I'd like to see a proof of concept of this in action but couldn't find one
  • I couldn't find an easy way to use git with NodeJS yet.
  • I'm not sure how to parse the returned diff on client apps developed with the IONIC framework, so AngularJS.

Note : The api will only be readable. All DB movement will be triggered by a custom web back-end used by few users.

Armel Larcier
  • 15,747
  • 7
  • 68
  • 89

1 Answers1

0

I used the ideas in that post for an experimental configuration-management service. That code is in Erlang and I can't offer Node-specific suggestions, but I have some general advice.

Calling out to git itself wasn't a great option at the time from any of the languages I was interested in using. Using git as a generic versioned-object store actually works surprisingly well, but using git plumbing commands is a pain (and slow, due to all of the forking) and there were (again, at the time) limitations to all of the available native git libraries.

I wound up implementing my own persistent trie data structure and put a git-like library interface atop it. The nice thing about doing this is that your diffs can be sensitive to the data format you're storing; if you call out to git, you're stuck with finding a serialization format for your data that is amenable to standard diffs. (Without a diffable format, though, you can still send back a sequence of operations to the client to replay on whatever stale objects they have.)

willb
  • 293
  • 1
  • 11