Can't read whole text file, if size bigger than ~10MB

Question

I am trying to read a text file line by line with Java and searching for special lines with regex. The problem is my file is around 28MB. There are 198546 lines in it, but after around 110000 lines, my script doesn't search and finish the script without any exception. I think this is relates to the BufferedReader size.

This is the one part of my code:

try {
    BufferedReader br = new BufferedReader(new FileReader(logsArrayList.get(i).toString()));
    String strLine;
    number = 0;
    while ((strLine = br.readLine()) != null) {
        if ((lineNumber % 2) == 0) {
            firstLine = strLine;
        }
        //control the regex 
        Matcher m = runRegex.matcher(strLine);
        if (((lineNumber % 2) != 0) && m.find()) {
            number++;
            stringList.add(number + ") " + firstLine);
            stringList.add(strLine);
            logList.add(logsArrayList.get(i).toString());
            logList.add(logsArrayList.get(i).toString());
        }
        publish(stringList.size());
        lineNumber++;
    }
        br.close();
} catch (Exception e) {
    System.err.println("Error: " + e.getMessage());
}

So does anyone have any idea about why I am not able to read the whole file? Is it related to the BufferedReader size, and if it is, how can I handle this problem?

Any help, ideas, or examples are appreciated.

EDIT: The script doesn't throw an exception.

does it throws any errors or any stack overflow occurs please put the stack trace with the code — Murali, Dec 10 '13 at 13:50
Can you post more of your code please? Also, this line is duplicated (`logList.add(logsArrayList.get(i).toString())`) You should probably remove one of those. — Elliott Frisch, Dec 10 '13 at 13:50
What's the exception? You probably just need to increase your VM size. — William Falcon, Dec 10 '13 at 13:50
You should remove the try-catch block to let your exception show. You shouldn't swallow exceptions like this. — sulai, Dec 10 '13 at 13:50
@sulai i have also try like that, remove try-catch but no any exceptions. — Batuhan B, Dec 10 '13 at 13:57
Ok, if it doesn't throw an exception, what *does* it do? What does "my script stops searching"? What is the exact behaviour? — dcsohl, Dec 10 '13 at 13:58
@dcsohl Sorry for conflict of my sentence. After the 110000 line my script doesn't search the rest of the file and finish like there is no any problem. I realize this problem when i count the lines in notepad++. — Batuhan B, Dec 10 '13 at 14:02
If your line has more than 1024 characters, notepad will break the line and count it as two lines even though there is no linebreak in the file. — mmaag, Dec 10 '13 at 14:08

score 1 · Answer 1 · edited May 23 '17 at 12:13

1

If you have any problem with stack size then you should increase the heap size

Just increase the heap size of Java

java -Xmx250m

If you running your project from IDE set -Xmx250m in arguments.

250m is 250mb

increasing vm size

edited May 23 '17 at 12:13

Community

1
1

answered Dec 10 '13 at 13:53

Murali

774
6
12

score 0 · Answer 2 · answered Dec 10 '13 at 13:51

0

If you are using Java 7, you can do it this way:

try (BufferedReader br = Files.newBufferedReader(Paths.get("path/to/your/file.txt"), StandardCharsets.UTF_8)) {
    for (String line = null; (line = br.readLine()) != null;) {
        //...
    }

}

answered Dec 10 '13 at 13:51

dstronczak

2,406
4
28
41

Can't read whole text file, if size bigger than ~10MB

2 Answers2