1

I'm working on an app that will generate a Json potentially very big. In my tests this was 8000 rows. This is because is an aggregation of data for a year, and is required to display details in the UI.

For example:

"voice1": {
    "sum": 24000,
    "items": [
        {
            "price": 2000,
            "description": "desc1",
            "date": "2021-11-01T00:00:00.000Z",
            "info": {
                "Id": "85fda619bbdc40369502ec3f792ae644",
                "address": "add2",
                "images": {
                    "icon": "img.png",
                    "banner": null
                }
            }
        },
        {
            "price": 2000,
            "description": "desc1",
            "date": "2021-11-01T00:00:00.000Z",
            "info": {
                "Id": "85fda619bbdc40369502ec3f792ae644",
                "address": "add2",
                "images": {
                    "icon": "img.png",
                    "banner": null
                }
            }
        }
    ]
},

The point is that I can have potentially 10 voices and for each dozen and dozens of items.

I was wondering if you can point to me some Best Practices or if you have some tips about them because I've got the feeling this can be done better.

Mirko Urru
  • 25
  • 1
  • 6

2 Answers2

1

It sounds like you are finding out that JSON is a rather verbose format (not as bad as XML but still very verbose). If you are worried about the size of messages between server client and you have a few options:

  1. JSON compresses rather well. You can see how most tokens repeat many times. So make sure to Gzip or Snappy before sending to clients. This will drastically reduce the size, but cost some performance for inflating / deflating.

  2. The other alternative is to not use JSON for transfer, but a more optimized format. One of the best options here is Flat Buffers. It does require you to provide schemas of the data that you are sending but is an optimized binary format with minimal overhead. It will also drastically speed up your application because it will remove the need for serialization / deserialization, which takes a significant time for JSON. Another popular, but slightly slower alternative is Protobuf.

Oswin Noetzelmann
  • 9,166
  • 1
  • 33
  • 46
0

The only thing immediately obvious to me is that you would likely want to make a list of voices (like you have for items) rather than voice1, voice2, etc.

Beyond that it really just depends the structure of the data you start with (to create the json) and the structure of the data or code at the destination (and possibly also the method of transferring data if size is a concern). If you're doing a significant amount of processing on either end to encode/decode the json that can suggest there's a simpler way to structure the data. Can you share some additional context or examples of the overall process?

Matthew Clark
  • 1