How to turn a doc with 10k independent JSON objects into a JSON array in Java

Question

I am currently working on migrating a database from a non-SQL source to an SQL database. The non-SQL source outputs the data in a JSON doc that is just a series of independent JSON objects. I an using JSONObject within Java, and that (to my understanding) can only recognize the top most object within the document. In order to get around this issue I am writing code to convert the independent objects into an array.

The current method I am using involves converting the JSON doc into a string, counting curly brackets to find objects, and then inserting them into an array.

for (int i = 0; i < doc.length(); i++) {
 char currentChar = doc.charAt(i);
 if (currentChar == '{') {
  Integer jsonStart = i;
  Integer openBrace = 1;
  Integer closeBrace = 0;
  while (openBrace > closeBrace) {
   i++;
   currentChar = doc.charAt(i);
   if (currentChar == '{') {
    openBrace++;
   }
   if (currentChar == '}') {
    closeBrace++;
   }
  }
  Integer jsonEnd = i;
  String currentString = doc.substring(jsonStart, jsonEnd + 1);
  JSONObject currentJSONObject = new JSONObject(currentString);
  returnJSONArray.put(currentJSONObject);

Due to size, the database had to be divided into multiple 10k object documents. The code worked well until one of the documents had braces stored within the value. So I added some code to watch for values and ignore those based on quotation marks beneath the close curly bracket counter.

if (currentChar == '"') {
 i++;
 currentChar = mongoExport.charAt(i);
 while (!(currentChar == '"')) {
  i++;
  currentChar = mongoExport.charAt(i);
 }

This worked for the document with value pairs that contained curly brackets, but upon testing it against the rest of the documents I experience a "String index out of range: big number" error in one of the other documents that traces back to the while loop looking for and end quotation mark. From what I can figure, this means that there are also values that contain quotation marks. I tried some code to check for escape characters before quotation marks, but that changed nothing. I can't check through these documents manually, they are far too long for that. Is there a way for me to handle these strings? Also, was there a far easier method I could have used that I was unaware of from the beginning?

You really don't need to manually parse the JSON. Are you allowed to use other libraries? Can you show a small sample of what the JSON looks like? — stdunbar, Dec 02 '22 at 17:12
You're trying to reinvent the wheel here. Use GSON or Jackson to parse the JSON text into objects. Once you have pojos, there are various options to store them in the SQL database (JPA/Hibernate, JDBC, Spring JdbcTemplate, etc). — E-Riz, Dec 02 '22 at 17:17
Take a look at [some](https://stackoverflow.com/questions/67982012/parse-json-array-java) [other](https://stackoverflow.com/questions/18977144/how-to-parse-json-array-not-json-object-in-android) posts. Do not try to parse the JSON directly. — stdunbar, Dec 02 '22 at 17:47
Using other libraries is difficult because of some VERY stringent company policy. If that is my best bet, then I'll put in the paperwork for it. I would love to share some of the JSON, but can't for similar reasons. The objects are all entirely overly complex though, often having arrays that store several other arrays in layers. I can process all of that, but I'll see what tool I'm allowed to get my hands on for parsing the JSON for me. — ZenubisSpyke, Dec 02 '22 at 18:01

stdunbar · Answer 1 · 2022-12-02T19:41:16.533

Even using the java.json package doesn't require manual parsing. something like:

import java.io.ByteArrayInputStream;
import java.io.InputStream;

import javax.json.Json;
import javax.json.JsonArray;
import javax.json.JsonObject;
import javax.json.JsonReader;
import javax.json.JsonValue;

...

private static final String jsonString = "[" +
        "{\n" +
        "\"id\":123,\n" +
        "\"name\":\"Bob Marley\",\n" +
        "\"address\":{\n" +
        "\"street\":\"123 Main St\",\n" +
        "\"city\":\"Anytown\",\n" +
        "\"state\":\"CO\",\n" +
        "\"zipcode\":80205\n" +
        "},\n" +
        "\"phoneNumbers\":[\"3032920200\"],\n" +
        "\"role\":\"Developer\"\n" +
        "},\n" +
        "{\n" +
        "\"id\":456,\n" +
        "\"name\":\"Tommy Tutone\",\n" +
        "\"address\":{\n" +
        "\"street\":\"456 Main St\",\n" +
        "\"city\":\"Sometown\",\n" +
        "\"state\":\"CO\",\n" +
        "\"zipcode\":80205\n" +
        "},\n" +
        "\"phoneNumbers\":[\"1238675309\"],\n" +
        "\"role\":\"Developer\"\n" +
        "}\n" +
        "]";

... 

@GET
@Produces("text/plain")
public String hello() {
    InputStream inputStream = new ByteArrayInputStream(jsonString.getBytes());
    JsonReader jsonReader = Json.createReader(inputStream);
    JsonArray jsonArray = jsonReader.readArray();

    for (JsonValue jsonValue : jsonArray) {
        JsonObject jsonObject = jsonValue.asJsonObject();

        System.out.println("next object id is " + jsonObject.getInt("id"));
        JsonObject addressObject = jsonObject.getJsonObject("address");

        System.out.println("next object city is " + addressObject.getString("city"));
    }

    return "Hello, World!";
}

This gets the first level objects (for example, "id") and nested objects ("address" in this example). I intentionally did not create a POJO type object that would represent the JSON object - you can do that but you'll have to decide if it's worthwhile to have a full object of your data or just pull it with things like getString().

How to turn a doc with 10k independent JSON objects into a JSON array in Java

1 Answers1