Java heap space of BlueJ

Question

I have a problem with the Java heap space of BlueJ. I have written a program which reads in a .txt to a String and goes through all the characters of the string and do some stuff(guess this is not really important). Some of the .txt are really large(around 200 million). If I try to execute the program with these .txt i get this "Exception in thread "AWT-EventQueue-0" java.lang.OutOfMemoryError: Java heap space" error code. I increased the bluej.windows.vm.args and bluej.windows.vm.args in the bluej.def to 8gb. And it still does not work. But I actually guess that even a 200million character String would not exceed this limit. Here is my code of how I read in the .txt

try
    {
        FileReader reader = new FileReader(input.getText());
        BufferedReader bReader = new BufferedReader(reader);
        String parcour = "";
        String line = bReader.readLine();
        while(line != null)
        {
            parcour += line;
            line = bReader.readLine();            
        }

input.getText() gets the file paths. I would be really grateful for an answer. Thanks :) - Cyaena

You are storing the entire file in the `parcour` variable. This will eat a lot of memory. Also, consider using a `StringBuilder` when appending data to the string — ar34z, Nov 28 '16 at 14:08
But actually a String should not eat that much memory, does not it? What advantage would I have to use a StringBuilder? And how do I set it up roughly? — Cyaena, Nov 28 '16 at 14:17
I'm not the right person to explain memory usage, but storing 200 mil. characters in memory doesn't sounds very promising. When concatenating Strings in a loop, a `StringBuilder` is faster. What are you trying to do with the 200 million lines? Perhaps this can help you: http://stackoverflow.com/questions/10202905/is-it-advisable-to-store-large-strings-in-memory-or-repeatedly-read-a-file More info on memory usage: http://www.javamex.com/tutorials/memory/string_memory_usage.shtml — ar34z, Nov 28 '16 at 14:27
Well, I actually would guess it is 200MB then, probably a bit more, but not something like 8GB. It is for a competition where are some examples. The smaller examples are not any problems. My algorithm is working fine, also with a runtime which is not too bad. But for these great files, yeah, I get this error. — Cyaena, Nov 28 '16 at 14:33
Perhaps if you explain your use case, others can help to solve the problem. Can't you process a single line in the while loop rather than storing everything in the variable? — ar34z, Nov 28 '16 at 14:36
Already tries this out. But I guess the .txt is just about one or at least not many lines. So the problem still remains. It actually would be possible here otherwise, yes. — Cyaena, Nov 28 '16 at 14:38

score 0 · Answer 1 · answered Nov 29 '16 at 12:19

In the below explanation only the plain memory for the data is in the scope. All additional memory need for the structures are left out. It's more an overview as an in deep detail view.

The memory is eaten at those lines

String parcour = "";
...
String line = bReader.readLine();
...
parcour += line;

The line parcour += line is compiled into the class file as

new StringBuilder().append(parcour).append(line).toString()

Assume parcour contains a string of size 10 MB and line would be of size 2 MB. Then the memory allocated during parcour += line; would be (roughly)

// creates a StringBuilder object of size 12 MB
new StringBuilder().append(parcour).append(line)

// the `.toString()` would generate a String object of size 12 MB
new StringBuilder().append(parcour).append(line).toString()

Your code needs before the newly created String is assigned to parcour around 34 MB.

parcour                            = 10 MB
the temporary StringBuilder object = 12 MB
the String fromStringBuilder       = 12 MB
------------------------------------------
total                                34 MB

A small demo snippet to show that the OutOfMemoryException is thrown much earlier then you currently expect.

OOMString.java

class OOMString {
    public static void main(String[] args) throws Exception {
        String parcour = "";
        char[] chars = new char[1_000];
        String line = new String(chars);
        while(line != null)
        {
            System.out.println("length = " + parcour.length());
            parcour += line;
        }        
    }
}

OOMStringBuilder.java

class OOMStringBuilder {
    public static void main(String[] args) throws Exception {
        StringBuilder parcour = new StringBuilder();
        char[] chars = new char[1_000];
        String line = new String(chars);
        while(line != null)
        {
            System.out.println("length = " + parcour.length());
            parcour.append(line);
        }        
    }
}

Both snippets do the same. They add a 1,000 charcater string to parcour till the OutOfMemoryException is thrown. To speed it up we limit the heap size to 10 MB.

output of java -Xmx10m OOMString

length = 1048000
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

output of java -Xmx10m OOMStringBuilder

length = 2052000
Exception in thread "main" java.lang.OutOfMemoryError: Java heap space

When you execute the code you will notice that OOMString needs much more time to fail (even at a shorter length) than OOMStringBuilder.

You also need to keep in mind that a single character is two bytes long. If your file contains 100 ASCII characters they consume 200 byte in memory.

Maybe this small demonstration could explain it a little bit for you.

Sander · Answer 2 · 2017-04-27T14:09:14.400

I've had some problems with BlueJ and Heap Space errors as well. In my case, opening the terminal crashed the entire application. I suspect this had something to do with generating a lot of output, similar to your large String. In my case, I accidentally had created an endless loop somewhere which broke the terminal window.

I had to remove all property files. Now BlueJ works again and gives no more OutOfMemoryErrors. I hope this might be helpful in other cases as well.

Java heap space of BlueJ

2 Answers2