How efficiently compare changes in JSON and keep track of it

Question

I have a very big nested object(stringify comes 4mb) similar as following which can be 15-20 level deep:

{
  "a": 5,
  "b": {
      "0": 1,
      "1": "x",
      "length": 2
   },
  "x": {
      "a": 1,
      "1": "z"
  },
  "c": {
      "0": 3,
      "1": "Am",
      "3": {
          "0": 3,
          "1": "x",
          "2": {
              "0": 3,
              "1": "Y",
              "length": 2
          },
          "length": 3
      },
      "length": 4
  }
}

After every action on page some property of object gets changed don't know what e.g. above becomes following(see c["3"]["2"]):

{
  "a": 5,
  "b": {
      "0": 1,
      "1": "x",
      "length": 2
   },
  "x": {
      "a": 1,
      "1": "z"
  },
  "c": {
      "0": 3,
      "1": "Am",
      "3": {
          "0": 3,
          "1": "x",
          "2": {z:5,n:"y"},
          "length": 3
      },
      "length": 4
  }
}

I can't store whole object every time as it will overflow the memory. So I want to keep track of changes only. So when replaying the action I'll just change that part of object and it will show new object.

Note: I don't want to go with comparison of object in iterative way as that computation will also cost me much as object size is very high.

I'm not getting any heads up how to start on it.

I think some way hashing object properties should help(just like angularJS does in its digest cycle).

Don't use blockquotes for emphasis. Only use blockquote when quoting something. — T.J. Crowder, Oct 01 '18 at 16:16
Why not store the actions?... sort of a "command" pattern approach. — Seth Flowers, Oct 01 '18 at 16:20
Yea, thats what I need but for every action I need what all props got changed in object. Are u suggesting iterating object is only the solution? — Akhilesh Kumar, Oct 01 '18 at 16:23
You might want to [Detecting change in a Javascript Object](https://stackoverflow.com/questions/11578377/detecting-change-in-a-javascript-object) and use this information to save the diff. — t.niese, Oct 01 '18 at 16:25
No Sir I can't edit original object every timestamp I get just value of new object no reference as they are send over postmessage to extension. But 1 thing for sure everytime most part(part of object schema) of object will be same. — Akhilesh Kumar, Oct 01 '18 at 16:35

score 1 · Answer 1 · answered Oct 01 '18 at 16:23

1

You have plenty of js libs that are able to calculate a diff between two JSON payloads, often under the form a patch that can be later applied

either on the original payload to get the new version
or on the latest version in a reverse mode to get back to original version.

For instance:

https://json-delta.readthedocs.io/en/latest/philosophy.html https://github.com/benjamine/jsondiffpatch

answered Oct 01 '18 at 16:23

TacheDeChoco

3,683
1
14
17

Does that work good with large size of JSON? Can u suggest one which has an algorithm which support JSON in MBs? – Akhilesh Kumar Oct 01 '18 at 16:40

binariedMe · Accepted Answer · 2019-02-19T03:32:07.250

1

I think if you plan earlier instead of using a library, you can optimze further. What I think is flatten your object in background. At each change state, just save diff as git does. Finally to revert back to a timestamp, just merge in reverse order and you get the final object.

What you should do is create one attribute like

"user.name.first" : "rohit"

And recursively create flatten object, create diff accordingly and then save the diff only.

Create original object of some timestamp, just merge recursively. You can memoisation to improve further.

edited Feb 19 '19 at 03:32

answered Oct 01 '18 at 17:00

binariedMe

4,309
1
18
34

Like "user.name.first". : "rohit" – Akhilesh Kumar Oct 01 '18 at 17:07

How efficiently compare changes in JSON and keep track of it

2 Answers2