How to remove key value pairs from a JSON file in java

Question

I was wondering if anyone can help me or hint me towards how to edit the attached dummy JSON file in Java.

As you can see I have a head object that contains many values and children that follow the same pattern.

I wanted to know if there was a way to remove all the keys where the value is -1.

Following is what I was trying based on many websites using jackson:

try {
            // create object mapper instance
            ObjectMapper mapper = new ObjectMapper();

            // convert JSON file to map
            Map<?, ?> map = mapper.readValue(Paths.get("test.json").toFile(), Map.class);

            // print map entries
            for (Map.Entry<?, ?> entry : map.entrySet()) {
                isInteger = main.isObjectInteger(entry.getValue());

                
//              System.out.println("if value is all: " + entry.getKey() + "=" + entry.getValue());
//

The above code will display the structure of the file, however my problem is reaching the -1 values inside the children and removing them.

Using the .getClass and .simpleName methods I know that it is an arrayList but I am confused as to how to search through it.

Any help will be greatly appreciated!

Don't use type bindings like `Map` on reading, because it is not guaranteed to get the same structure back on writing. Use `JsonNode` or whatever Jackson uses for JSON tree representation, walk it recursively into JSON objects and arrays, find the elements you're going to delete, and simply delete them all. That's it. — terrorrussia-keeps-killing, Dec 03 '20 at 07:31
[Please do not upload images of code/errors when asking a question.](//meta.stackoverflow.com/q/285551) — Andreas, Dec 03 '20 at 07:34
You can try Gson library if the jackson is not necessary. It is much easier (for me). Just create classes that contains json fields that you need. Then gson will parse it as your class object. — stuck, Dec 03 '20 at 08:40

score 4 · Accepted Answer · answered Dec 04 '20 at 00:49

In Jackson you can read whole JSON payload as JsonNode and iterate over all properties check given condition. In case condition is met you can remove given field. To do that you need to implement recursive method. Take a look on below example:

import com.fasterxml.jackson.databind.JsonNode;
import com.fasterxml.jackson.databind.ObjectMapper;
import com.fasterxml.jackson.databind.SerializationFeature;
import com.fasterxml.jackson.databind.node.ArrayNode;
import com.fasterxml.jackson.databind.node.ObjectNode;

import java.io.File;
import java.io.IOException;
import java.util.Iterator;
import java.util.Map;
import java.util.Objects;
import java.util.function.Predicate;

public class JsonRemoveSomeFieldsApp {

    public static void main(String[] args) throws IOException {
        File jsonFile = new File("./resource/test.json").getAbsoluteFile();

        ObjectMapper mapper = new ObjectMapper();
        mapper.enable(SerializationFeature.INDENT_OUTPUT);
        JsonNode root = mapper.readTree(jsonFile);

        JsonCleaner jsonCleaner = new JsonCleaner(root, (node) -> node.isNumber() && node.numberValue().intValue() == -1);
        JsonNode result = jsonCleaner.removeAll();

        // write to file
        mapper.writeValue(System.out, result);
    }
}

class JsonCleaner {

    private final JsonNode root;
    private final Predicate<JsonNode> toRemoveCondition;

    JsonCleaner(JsonNode node, Predicate<JsonNode> toRemoveCondition) {
        this.root = Objects.requireNonNull(node);
        this.toRemoveCondition = Objects.requireNonNull(toRemoveCondition);
    }

    public JsonNode removeAll() {
        process(root);
        return root;
    }

    private void process(JsonNode node) {
        if (node.isObject()) {
            ObjectNode object = (ObjectNode) node;
            Iterator<Map.Entry<String, JsonNode>> fields = object.fields();
            while (fields.hasNext()) {
                Map.Entry<String, JsonNode> field = fields.next();
                JsonNode valueToCheck = field.getValue();
                if (valueToCheck.isContainerNode()) {
                    process(valueToCheck);
                } else if (toRemoveCondition.test(valueToCheck)) {
                    fields.remove();
                }
            }
        } else if (node.isArray()) {
            ArrayNode array = (ArrayNode) node;
            array.elements().forEachRemaining(this::process);
        }
    }
}

For below JSON payload:

{
  "name": "Head",
  "missed": -1,
  "covered": -1,
  "children": [
    {
      "name": "project1",
      "missed": -1,
      "covered": -1,
      "children": [
        {
          "name": "project1",
          "missed": 10,
          "covered": 11
        }
      ]
    },
    {
      "name": "project1",
      "missed": -1,
      "covered": 12,
      "children": [
        {
          "name": "project1",
          "missed": 10,
          "covered": -1
        }
      ]
    }
  ]
}

above code prints:

{
  "name" : "Head",
  "children" : [ {
    "name" : "project1",
    "children" : [ {
      "name" : "project1",
      "missed" : 10,
      "covered" : 11
    } ]
  }, {
    "name" : "project1",
    "covered" : 12,
    "children" : [ {
      "name" : "project1",
      "missed" : 10
    } ]
  } ]
}

See also:

Flattening a 3 level nested JSON string in java

thank you for the explanation and example, I will try this out! — Anika, Dec 04 '20 at 09:44

AnatolyG · Answer 2 · 2020-12-03T16:17:36.097

There are two main techniques to parse and generate JSON data (as well as many other formats like XML etc): object mapping and event/token/stream-oriented processing. The second way is the best way for many cases, including filtering. Props:

the file/data doesn't require to be loaded entirely into memory, you can process megs/gigs with no problems
it works much more faster, especially for large files
it's easy to implement any custom type/rule of transformation with this pattern

Both Gson and Jackson supports stream-oriented processing. To illustrate the idea here is just an example using a tiny parser/generator https://github.com/anatolygudkov/green-jelly

import org.green.jelly.AppendableWriter;
import org.green.jelly.JsonBufferedWriter;
import org.green.jelly.JsonEventPump;
import org.green.jelly.JsonNumber;
import org.green.jelly.JsonParser;

import java.io.StringWriter;

public class UpdateMyJson {
    private static final String jsonToUpdate = "{\n" +
            "\"name\": \"Head\",\n" +
            "\"missed\": -1,\n" +
            "\"children\": [\n" +
            "    {\n" +
            "        \"name\": \"project1\",\n" +
            "        \"fixes\": 0,\n" +
            "        \"commits\": -1,\n" +
            "    },\n" +
            "    {\n" +
            "        \"name\": \"project2\",\n" +
            "        \"fixes\": 20,\n" +
            "        \"commits\": 5,\n" +
            "    }\n" +
            "]\n" +
            "}";

    public static void main(String[] args) {
        final StringWriter result = new StringWriter(); // you can use FileWriter

        final JsonParser parser = new JsonParser();
        parser.setListener(new MyJsonUpdater(new AppendableWriter<>(result)));
        parser.parseAndEoj(jsonToUpdate); // if you read a file with a buffer,
        // to don't load the whole file into memory,
        // call parse() several times (part by part) in a loop until EOF
        // and then call .eoj()

        System.out.println(result);
    }

    static class MyJsonUpdater extends JsonEventPump {
        MyJsonUpdater(final JsonBufferedWriter output) {
            super(output);
        }

        @Override
        public boolean onNumberValue(final JsonNumber number) {
            if (number.mantissa() == -1 && number.exp() == 0) {
                return true; // return immediately
            }
            return super.onNumberValue(number); // otherwise pump the value to the result JSON
        }
    }
}

Just to clarify: is your library aimed to modify JSON streams on fly without an intermediate tree representation? — terrorrussia-keeps-killing, Dec 03 '20 at 21:29
@fluffy the lib is aimed to parse and generate JSON in stream/event manner with no any intermediate or any other object mapping/representation. This is like SAX/StAX for XML. And the lib shouldn't do more than required for this and should be GC-free. Of course, it's easy to convert an input sequence of events/tokens to an output one with some transformations like, for example, filtered/skipping or other stateless (almost) transformations, when you don't really need the file to be entirely or even partially loaded into memory. — AnatolyG, Dec 04 '20 at 20:02
It's common place do discuss big-O issues for collections, you know, ArrayList vs LinkedList etc. And after that, when someone tries to extract/modify/filter 1 value from JSON with an object mapper or, for example, JsonPath wasting 1000x more CPU time/memory than it could be, my engineering's eyes are bleeding))) — AnatolyG, Dec 04 '20 at 20:19
Cool. Totally agree with the last point for suboptimal use of JsonPath or other tools like that. The most misinterpreted thing here is that people usually do not care about this, and load entire documents first to string, and then - to a JSON tree (or worse -- use type binding that can break the original document due to possible asymmetric type binding for read and write respectively). The most used tools are not that suitable for this kind of purpose, but they offer a bit of easy to do things. — terrorrussia-keeps-killing, Dec 05 '20 at 01:18
The reason of why I suggested to use JSON tree representation in my very first comment before you answered the question is that this kind of representation is easy for beginners to understand and use. I have never heard of anything like SAX/StAX for JSON like your library, and built similar streaming stuff on top of Gson and Jackson myself. Not sure how good these libs are at parsing-only, but this was way better than trying to load huge documents and put a server instance down. So yeah, your solution is really cool and does the thing right. — terrorrussia-keeps-killing, Dec 05 '20 at 01:22

How to remove key value pairs from a JSON file in java

2 Answers2

Linked