parsing to json using jackson-mapper causes

Question

I have a big object I want to parse to json Jackson-mapper. It works fine, till the object becomes too big.

I'm using intellij on mac.

code:

private String serializeToJson(T item) {
    String json;
    ObjectWriter ow = new ObjectMapper().writer().withDefaultPrettyPrinter();
    try {
        json = ow.writeValueAsString(item);
    } catch (IOException e) {
        e.printStackTrace();
        json = "";
    }
    return json;
}

error:

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
:BL_generate FAILED

BUILD FAILED

Total time: 12 mins 14.491 secs
    at java.util.Arrays.copyOfRange(Arrays.java:3664)
    at java.lang.String.<init>(String.java:201)
    at java.lang.StringBuilder.toString(StringBuilder.java:407)
    at org.codehaus.jackson.util.TextBuffer.contentsAsString(TextBuffer.java:362)
    at org.codehaus.jackson.io.SegmentedStringWriter.getAndClear(SegmentedStringWriter.java:100)
    at org.codehaus.jackson.map.ObjectWriter.writeValueAsString(ObjectWriter.java:394)
    at com.waze.routing.automation.io.string.JsonFileHandler.serializeToJson(JsonFileHandler.java:81)

I'm not sure how to split an object or to write it in parts (append to existing file)

i have tried to enlarge heap space, but it didn't help

task BL_generate(type: JavaExec) {
    jvmArgs = ["-Xms1024m","-Xmx1024m"]
    classpath sourceSets.main.runtimeClasspath
    main = "com.m.BaselineGeneratorRunner"
}

i have read few posts: post1, post2

but i'm not sure how to use it in my case. How can streaming help me with one big object? (not array).

Why you are using the old version of Jackson? `org.codehaus.jackson` move to the newest version and check again. — igreenfield, Apr 27 '15 at 16:55
[search maven](http://search.maven.org/#search%7Cga%7C1%7Cg%3A%22com.fasterxml.jackson.core%22) — igreenfield, Apr 28 '15 at 01:15
i searched via maven repository, i see codehaus is more used. http://mvnrepository.com/search?q=jackson — Elad Benda, Apr 28 '15 at 12:15
Yes poeple doesn't move yet to the new artifcts look at jackson project page — igreenfield, Apr 28 '15 at 16:07

score 2 · Answer 1 · answered Apr 27 '15 at 16:51

The main question here is why do you want a java.lang.String here? Usually you would rather write the contents to a stream or File.

Streaming will not help you if you really want the whole JSON serialization as a String -- String object takes plenty of memory; at least 2x as much as contents as File would take.

Actually it is bit more than that: at the point where String is being constructed, the buffered version (in char[] sergments) takes as much space, so it is about 4x as much as equivalent File. Your heap is also divided into different areas (young, old generations), so I would guess that in worst case you may need about 10x as memory. If so, setting of 1024 megs should let you handle JSON content of about 100 megabytes.

I would try to figure out how to avoid having to create a big String first, and then the problem should be solvable.

i'm not sure I understand the meaning of "json streaming". how is it economic? — Elad Benda, Apr 28 '15 at 07:28
JSON streaming fundamentally means reading/writing tokens incrementally, one at a time, and not necessarily reading the whole document in at once (or having to construct content all in memory before writing anything). If it is possible to do this (for example, just read subsets of content at given time), it can help a lot with memory usage. There may be some confusion also from the fact that all Jackson parsing/generation uses underlying streaming parser/generator; even when usage at high level is not incremental/streaming. — StaxMan, Apr 29 '15 at 01:30

parsing to json using jackson-mapper causes

1 Answers1

Linked