Fastest way of reading large object-text file

Question

I store over 40,000 objects into a text file. My problem is reading all objects from text file is too slow. It even takes 4349 ms for 1,000 objects-text file.

This is reading objects from text file.

long startR = System.currentTimeMillis();

try{
    ois = new ObjectInputStream(new FileInputStream(f));
    code_from_file.clear();
    Codes obj = new Codes();

    while( (obj = (Codes) ois.readObject()) != null){   
        if(obj instanceof Codes){
            code_from_file.add(obj);
        }
    }

}catch (EOFException ex){ 

} catch (ClassNotFoundException ex) {
    ex.printStackTrace();
} catch (FileNotFoundException ex) {
    ex.printStackTrace();
} catch (IOException ex) {
    ex.printStackTrace();
} finally{
    try {
        if (ois != null){
            ois.close();
        }
    } catch (IOException ex){
        ex.printStackTrace();
    }
}

long endR = System.currentTimeMillis();
System.out.println("Read code from file : " + (endR - startR) + "ms");

Is there any faster way for solving this problem?

Try to wrap it with an bufferedinputstream. ois = new ObjectInputStream(new BufferedInputStream(new FileInputStream(f))); — Eddie, Oct 20 '15 at 03:59

score 1 · Answer 1 · answered Oct 20 '15 at 04:03

An easy optimization to try would be to add buffering to your input stream processing. As you wrote it, each read may be hitting your disk. If you do fewer physical reads (in larger chunks), you may see a performance improvement.

int bufferSize = 16 * 1024;
ois = new ObjectInputStream(new BufferedInputStream(new FileInputStream(f), bufferSize));

When bytes are read via the ObjectInputStream, they are actually being read in memory from the BufferedInputStream's buffer. When this buffer becomes empty (has all been read), the BufferedInputStream will re-fill the buffer in one large read from the FileInputStream.

You can experiment with different buffer sizes to determine an appropriate balance between the number of I/O operations vs. the memory overhead.

Saravana · Answer 2 · 2015-10-20T04:16:22.820

Try using NIO, it has many enhancements and buffering

    RandomAccessFile aFile = new RandomAccessFile("test.txt", "r");
    FileChannel inChannel = aFile.getChannel();
    ByteBuffer buffer = ByteBuffer.allocate(1024);
    int read;
    while((read = inChannel.read(buffer)) != -1){
        os.write(buffer, 0, read);
    }
    inChannel.close();
    aFile.close();

above code is with the fixed width of buffer size

Fastest way of reading large object-text file

2 Answers2