0

I have a requirement where I have a huge response json which contains images which are passed into json as base64 data. There are many of these base64 string data in the json, each of which are huge, as these images are also of size 5-10 Mb. Hence the json size becomes large. I want to know if there is any way to convert this huge json to multipart-data or if there is any way to break this huge response json and send them in multiple parts in a single http request. Can anyone please provide some help on the above.

Akash Raveendran
  • 559
  • 1
  • 9
  • 22
  • The question, as is, is probably too broad. Could you specify what would be acceptable trade-offs for you ? E.g. is changing the response content type to MIME/Multipart an option ? Is providing links to the images instead of the images themselves an option ? There is a world of possibilities, but you'd be better of by editing your question and be clear about what may or may not be acceptable. – GPI Feb 16 '18 at 08:33
  • Ya, i want to send the response as a multipart so, response content type would be MIME/Multipart. No, I dont have an option to send image as link. I have to send it as base64 itself. – Akash Raveendran Feb 16 '18 at 09:32

2 Answers2

2

A few options, but might be easiest to use Jackson, a JSON parser for Java. Jackson can split a large JSON object into multiple, smaller objects that can be sent to the client. Here's a tutorial.

See this Stack Overflow response for other options.

tskittles
  • 126
  • 1
  • 2
  • 9
  • Thanks for the response @tskittles .Ya, we can split the json but how can send this large json as response, after splitting them into multiple smaller objects. This large json needs to be sent in a single http transaction. – Akash Raveendran Feb 16 '18 at 08:36
  • What's the point of splitting up the JSON if you want to send the JSON in a single http transaction? In any case, it is not possible to send multiple JSON objects since an HTTP response has only one body. – tskittles Feb 16 '18 at 08:49
  • Ya true, thats why I asked if there is a way to convert the json into a multipart-data, so that I can send it in a single response. – Akash Raveendran Feb 16 '18 at 09:11
0

I have a problem with your question. You want to split the huge response json, but still want one single request. At the HTTP protocol level, one single request means one single response. You can then nest the data the way you want using different container formats, it will not magically split the response but will only add some overhead.

You do have options at low protocol levels. One is to compress the response payload. Many frameworks and servlet containers can do it provided it is supported by the browser and all major browsers do nowadays. Another one is to chunk the reponse payload. At at higher level it is still one single stream of bytes, but it is send on the network in different chunks. It may be more robust depending on the network, mainly if you use several proxies or routers.

And the other option is at high level: just have the client issue a bunch of requests, each response containing a part of the large json.

Serge Ballesta
  • 143,923
  • 11
  • 122
  • 252
  • In a more simpler way, how do I send a huge json response over http. When I sent it directly, many a times due to its huge size, the response used to fail. Thats why I thought of going for multipart-data. Anyways, can you suggest a low-drop rate method to send a huge json as response over http. – Akash Raveendran Feb 16 '18 at 09:41
  • @AkashRaveendran: that's what *chunking* is made for. You really should dig into your system to search how you could send the huge response in chunks. Google for *chunk* and you servlet containe of framework. – Serge Ballesta Feb 16 '18 at 10:08
  • Ok. I will look into the *chunking* that you told. Thanks. – Akash Raveendran Feb 16 '18 at 10:13