Reading certain number of bytes into ByteBuffer

Question

I have a binary file of 10MB. I need to read it in chunks of different size (e.g 300, 273 bytes). For reading I use FileChannel and ByteBuffer. Right now for each iteration of reading I allocate new ByteBuffer of size, that I need to read.

Is there possible to allocate only once (lets say 200 KB) for ByteBuffer and read into it (300 , 273 bytes etc. )? I will not read more than 200KB at once. The entire file must be read.

UPD

public void readFile (FileChannel fc, int amountOfBytesToRead)
{
    ByteBuffer bb= ByteBuffer.allocate(amountOfBytesToRead);
    fc.read(bb);
    bb.flip();
    // do something with bytes
    bb = null;
}

I can not read whole file at once due to memory constraints. That's why I performing reading in chunks. Efficiency is also very important (that is why I don't want to use my current approach with multiple allocations). Thanks

Are you saying you want to read 300 bytes, then 273 bytes, then XYZ bytes from the `FileChannel`, but only allocate the `ByteBuffer` once? — Sotirios Delimanolis, Dec 11 '14 at 18:48
If it was a fixed amount of bytes to be reserved then you could call it once, have the read part in some sort of loop and then be done. — Emz, Dec 11 '14 at 18:49

user207421 · Accepted Answer · 2014-12-12T00:11:24.063

2

Declare several ByteBuffers of the sizes you need and use scatter-read: read(ByteBuffer[] dsts, ...).

Or forget about NIO and use DataInputStream,readFully(). If you put a BufferedInputStream underneath you won't suffer any performance loss: it may even be faster.

edited Dec 12 '14 at 00:11

answered Dec 11 '14 at 21:26

user207421

305,947
44
307
483

I don't want to read the whole file at once. In my program I'm very concerned about memory and efficiency. Allocating so much space will lead to garbage collection (I want to avoid it). Many allocations are costly as well (in terms of efficiency) – Sparrow_ua Dec 11 '14 at 21:34
You should state all those additional constraints in your question. But I didn't say anything about allocating them repeatedly, and you only mention a total of 200kB per read, which isn't all that much. – user207421 Dec 11 '14 at 21:46
@EJP I didn't know DataInputStream.readFully and have oversight it in your answer. So I deleted my superflous answer. At least I have learned something:) – Gren Dec 12 '14 at 00:51
@EJP Thanks. I switched to combination DataInputStream with BufferedInputStream as you said. My function that does reading and parsing works 30ms faster(compared to NIO approach) for 4Mb file. That difference is important! And in terms of memory it will be more efficient. Since I read very small chunks! – Sparrow_ua Dec 12 '14 at 19:22

Reading certain number of bytes into ByteBuffer

1 Answers1