OpenGL VBO from array on SD Card (Android)

Question

I parse a binary point cloud file and get large arrays of points and their properties. These more often then not run out of memory when allocating towards VBOs before drawing. I need a solution that handles the memory problem but is still fast enough to feed the buffers. Since the application memory space is too small, could I somehow write a 'file' to the SD Card and populate the buffers piece-wise from there? I can't parse the file more than once.

Crashing lines:

-->ByteBuffer vbb = ByteBuffer.allocateDirect(lasVertices.length * 4);
vbb.order(ByteOrder.nativeOrder());
vertexBuffer = vbb.asFloatBuffer();
vertexBuffer.put(lasVertices);
vertexBuffer.position(0);

Ani · Accepted Answer · 2011-07-07T00:52:23.523

3

Have you looked at using MappedByteBuffer? This should allow you to parse the file once and write a memory optimized binary version to sdcard and read it back in chunks, fast.

// Open the file here, store it in a field
RandomAccessFile file = null;
try {
    mFile = new RandomAccessFile("/sdcard/reallyLargeFile.dat", "r");
} catch (FileNotFoundException e) {
    e.printStackTrace();
}

Where you want to render, do this:

for (int offset = 0; offset < POINTS_SIZE; offset += BLOCK_SIZE)
if (file != null)
    try {
        MappedByteBuffer buffer = file.getChannel().map(FileChannel.MapMode.READ_ONLY, offset, BLOCK_SIZE);
        vertexBuffer.put(buffer);

        vertexBuffer.position(0);

        // Render here
    } catch (IOException e) {
        e.printStackTrace();
    }

where POINTS_SIZE = total number of points * 4 ( = size of float) and BUFFER_SIZE is a (divisible) size that doesn't crash your application. Note that this is a single threaded solution and it might be possible to amortize your cost by using multiple threads reading smaller buffers simultaneously. Also, while this might in practice turne out to be just as efficient as a bunch of seeks and reads, it yields far more elegant and maintainable code. It is also entirely possible that memory mapped files may provide a performance boost on many machines.

Hope this helps.

edited Jul 07 '11 at 00:52

answered Jul 06 '11 at 15:04

Ani

10,826
3
27
46

You don't happen to have an example of how I would do this do you? – RedLeader Jul 06 '11 at 15:39
I will edit my post later this evening and post a code sample. – Ani Jul 06 '11 at 19:52
Thank you I do really appreciate your time. What I am doing now is parsing my binary file, and storing float arrays in a new file of only the data I need (I'll never parse that file again), In "frames" of 1000 points (8kbytes). Then I can Map to that file and hopefully do a Map.load and allocate from that load super fast. Still, I'm looking forward to what you did! – RedLeader Jul 06 '11 at 22:50
Thanks for the code! However, vertexBuffer.put(buffer) won't work because it is a MappedByteBuffer... – RedLeader Jul 07 '11 at 17:43
It will, because [MappedByteBuffer](http://developer.android.com/reference/java/nio/MappedByteBuffer.html) derives from ByteBuffer and [put](http://developer.android.com/reference/java/nio/ByteBuffer.html#put%28java.nio.ByteBuffer%29) takes a ByteBuffer. – Ani Jul 07 '11 at 18:09
Okay. I think I forced it... I do it a couple time, (check like 43). It is not displaying any of the points at this time. [link](pastebin.com/1YiU93jv) – RedLeader Jul 07 '11 at 18:33
It is probably not necessary to set points in the vertexBuffer using a .asFloatBuffer() - the underlying data is the same. Also, I suppose your file is organized like so a) x0 y0 z0 x1 y1 z1 ... x999 y999 z999 a0 r0 g0 b0 a1 r1 g0 b1 ... a999 r999 g999 b999 and not b) x0 y0 z0 a0 r0 g0 b0 x1 y1 z1 a1 r1 g1 b1 ... x999 y999 z999 a999 r999 g999 b999 – Ani Jul 07 '11 at 20:11
i*RECORDS_PER_FRAME*8 should probably be i*RECORDS_PER_FRAME*8 * 4 (= size of float in bytes). Ditto for all other sizes/offsets in your loops. Also, You're passing in ARGB colors, have you confirmed that OpenGL is interpreting it as such and not RGBA? I see that you're not specifying a stride parameter in glColorPointer. Using a stride (= 3 * 4), you can interleave colors and vertices (layout b) in the same array decreasing the number of maps you need. – Ani Jul 07 '11 at 20:11
@RedLeader let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/1242/discussion-between-ananthonline-and-redleader) – Ani Jul 07 '11 at 20:14

OpenGL VBO from array on SD Card (Android)

1 Answers1