Using Dynamic Buffers? Java

Question

In Java, I have a method

public int getNextFrame( byte[] buff )

that reads from a file into the buffer and returns the number of bytes read. I am reading from .MJPEG that has a 5byte value, say "07939", followed by that many bytes for the jpeg.

The problem is that the JPEG byte size could overflow the buffer. I cannot seem to find a neat solution for the allocation. My goal is to not create a new buffer for every image. I tried a direct ByteBuffer so I could use its array() method to get direct access to the underlying buffer. The ByteBuffer does not expand dynamically.

Should I be returning a reference to the parameter? Like:

public ByteBuffer getNextFrame( ByteBuffer ref )

How do I find the bytes read? Thanks.

score 1 · Accepted Answer · edited Oct 17 '12 at 03:49

1

java.io.ByteArrayOutputStream is a wrapper around a byte-array and enlarges it as needed. Perhaps this is something you could use.

Edit:
To reuse just call reset() and start over...

edited Oct 17 '12 at 03:49

Abdul Rahman

2,097
4
28
36

answered Mar 01 '11 at 17:45

Boris

4,327
2
19
27

The trick is making it reusable. ;) However you could create your own which is re-usable. – Peter Lawrey Mar 01 '11 at 17:49
1

You can reuse ByteArrayOutputStream, just call [reset()](http://download.oracle.com/javase/1.4.2/docs/api/java/io/ByteArrayOutputStream.html#reset%28%29)... – Boris Mar 01 '11 at 18:02
... however toByteArray() still takes a copy of the result which might be undesirable. – Peter Lawrey Mar 01 '11 at 18:07
You can extend ByteArrayOutputStream to get the buffer directly, it's a defined part of the class, just protected. You would have to synchronize yourself on it (if needed) and remember that this is the full array, including the bytes not yet written. But that's something one could handle if the memory-usage/performance matters. You'll get similar restrictions with other approaches too. – Boris Mar 01 '11 at 18:15
Alright. So java.io.ByteArrayOutputStream increases dynamically; then to avoid using toByteArray(), which copies the buffer, I need to extend the class. Thanks for input! Much more than I expected (i'm new here) -- impressive. – LJWolfe Mar 01 '11 at 21:17

score 1 · Answer 2 · edited Oct 17 '12 at 05:48

1

Just read the required number of bytes. Do not use read(buffer), but use read(buffer,0,size). If there are more bytes, just discard them, the JPG is broken anyway.

edited Oct 17 '12 at 05:48

Abdul Rahman

2,097
4
28
36

answered Mar 01 '11 at 17:48

Daniel

27,718
20
89
133

score 0 · Answer 3 · edited Oct 17 '12 at 05:57

EDIT:

Allocating a byte[] is so much faster than reading from a file or a socket, I would be surprised it will make much difference, unless you have a system where micro-seconds cost money.

The time it takes to read a file of 64 KB is about 10 ms (unless the file is in memory)

The time it takes to allocate a 64 KB byte[] is about 0.001 ms, possibly faster.

You can use apache IO's IOBuffer, however this expands very expensively.

You can also use ByteBuffer, the position() will tell you how much data was read.

If you don't know how big the buffer will be and you have a 64-bit JVM you can create a large direct buffer. This will only allocate memory (by page) when used. The upshot is that you can allocate a 1 GB but might only ever use 4 KB if that is all you need. Direct buffer doesn't support array() however, you would have to read from the ByteBuffer using its other methods.

Another solution is to use an AtomicReference<byte[]> the called method can increase the size as required, but if its large enough it would reuse the previous buffer.

score -1 · Answer 4 · answered Mar 01 '11 at 17:52

The usual way of accomplishing this in a high-level API is either let the user provide an OutputStream and fill it with your data (which can be a ByteArrayOutputStream or something completely different), or have an InputStream as return value, that the user can read to get the data (which will dynamically load the correct parts from the file and stop when finished).

Using Dynamic Buffers? Java

4 Answers4