Storing JSON document with AppEngine

Question

I'm trying to store JSON document into the AppEngine datastore using Objectify as persistence layer. To be able to query for document values, instead of just inserting the whole document as String field, I created a MapEntity which looks like this:

@Entity(name="Map") 
public class MapEntity {
    @Id
    private Long id;
    private Map<String,String> field;
        // Code omitted 
}

Since eventually when "unrolled" every key-value in the JSON document can be represented with Map

Example:

    String           subText          = "{\"first\": 111, \"second\": [2, 2, 2], \"third\": 333}";
    String           jsonText         = "{\"first\": 123, \"second\": [4, 5, 6], \"third\": 789, \"fourth\":"
                                        + subText + "}";

I will have the map fields stored in the datastore:

KEY              VALUE
field.first  => 123
field.second => [4,5,6]
field.third  => 789
field.fourth-first => 111
field.fourth-second => [2,2,2]
field.fourth-third => 333

If I use my parse() method:

Parse the JSON document using JSON.Simple library and then do a recursive parse:

private MapEntity parse(String root, MapEntity entity, Map json) {
    Iterator iter = json.entrySet().iterator();
    while (iter.hasNext()) {
         Map.Entry entry = (Map.Entry) iter.next();
         if (entry.getValue() instanceof Map){
             entity = parse((String)entry.getKey()+"-", entity, (Map) entry.getValue());
             System.out.println("Map instance");
         } else {
             entity.setField(root + String.valueOf(entry.getKey()), String.valueOf(entry.getValue()));  
         }
    }
    return entity;
}

My app works like this:

 MapEntity jsonEntity = new MapEntity();
 Map json = null;
 json = (Map) parser.parse(jsonText, containerFactory); // JSON.Simple parser
 jsonEntity = parse("", jsonEntity, json);

Problems I encounter are:

• I can't use the "." dot in the Map key field, so I have to use the "-"
• Also my approach in storing JSON document is not very efficient

Answer 1

If your JSON follows a strict format, you'd probably be better off constructing a class to represent your data format and serializing directly to and from that class using a library like Jackson. You can use that class directly as your entity class in Objectify, but whether you want to do depends on whether you want to:

• Store and expose the exact same set of data
• Tightly couple your storage and JSON representations

Answer 2

You could use JSONObject as a replacement for your MapEntity and store the json to google app engine as a string using the toString() method. Upon retrieval you could simply restore the JSONObject using the appropriate constructor. This, of course, limits your ability to index properties in app engine and query against them.

If you want Objectify to do this for you, you could register a Translator to take care of calling the toString() and reconstruction.