Converting RGBA to ARGB (glReadPixels -> AVAssetWriter)

Question

I want to record images, rendered with OpenGL, into a movie-file with the help of AVAssetWriter. The problem arises, that the only way to access pixels from an OpenGL framebuffer is by using glReadPixels, which only supports the RGBA-pixel format on iOS. But AVAssetWriter doesn't support this format. Here I can either use ARGB or BGRA. As the alpha-values can be ignored, I came to the conclusion, that the fastest way to convert RGBA to ARGB would be to give glReadPixels the buffer shifted by one byte:

UInt8 *buffer = malloc(width*height*4+1);
glReadPixels(0, 0, width, height, GL_RGBA, GL_UNSIGNED_BYTE, buffer+1);

The problem is, that the glReadPixels call leads to a EXC_BAD_ACCESS crash. If I don't shift the buffer by one byte, it works perfectly (but obviously with wrong colors in the video-file). What's the problem here?

Can't you instead allocate size+4, send buffer+4 into glReadPixels call, and then buffer+3 to AVAssetWriter? That will keep the GL-call aligned which may be a requirement on iOS. — Ville Krumlinde, Jan 07 '12 at 17:49

score 5 · Accepted Answer · answered Jan 07 '12 at 18:35

5

I came to the conclusion, that the fastest way to convert RGBA to ARGB would be to give glReadPixels the buffer shifted by one byte

This will however shift your alpha values by 1 pixel as well. Here's another suggestion:

Render the picture to a texture (using a FBO with that texture as color attachment). Next render that texture to another framebuffer, with a swizzling fragment shader:

#version ...
uniform sampler2D image;
uniform vec2 image_dim;
void main() 
{
    // we want to address texel centers by absolute fragment coordinates, this
    // requires a bit of work (OpenGL-ES SL doesn't provide texelFetch function).
    gl_FragColor.rgba = 
        texture2D(image, vec2( (2*gl_FragCoord.x + 1)/(2*image_dim.y), 
                               (2*gl_FragCoord.y + 1)/(2*image_dim.y) )
        ).argb; // this swizzles RGBA into ARGB order if read into a RGBA buffer
}

answered Jan 07 '12 at 18:35

datenwolf

159,371
13
185
298

1

As I mentioned, I don't need the alpha-information (I'm producing a h264-video). But that swizzling in an OpenGL shader is a great idea! :) – Dominik Seibold Jan 07 '12 at 19:22
2

Btw. I just found something very interesting: AVAssetWriter encodes video much (!) faster when specifying BGRA instead of ARGB. So the whole question really is about converting RGBA->BGRA instead of RGBA->ARGB and this pixel-swizzling-technique is absolutely mandatory, because the by me proposed one-byte-shifting technique can't convert RGBA->BGRA! – Dominik Seibold Jan 10 '12 at 02:49
@DominikSeibold please post some sample code for your implementation – anuj Apr 23 '14 at 03:42

score 1 · Answer 2 · answered Jan 07 '12 at 17:20

1

What happens if you put an extra 128 bytes of slack on the end of your buffer? It might be that OpenGL is trying to fill 4/8/16/etc bytes at a time for performance, and has a bug when the buffer is non-aligned or something. It wouldn't be the first time a performance optimization in OpenGL had issues on an edge case :)

answered Jan 07 '12 at 17:20

StilesCrisis

15,972
4
39
62

I even tried to double the buffer-size, but it still crashes. If I shift it by 4 bytes or one row (4*width) it works, but of course this doesn't help with initial problem. – Dominik Seibold Jan 07 '12 at 17:31
1

After your malloc, try adding NSLog(@"Buffer at 0x%08X", buffer); . Then look at the address that EXC_BAD_ACCESS occurs at. If the bad access is on the very start of your buffer, then probably OpenGL is writing memory in a way that blows up on unaligned access on ARM. I don't know enough about ARM to know what fails when it's unaligned (I'm more of a Mac OS guy at heart). Of course, if it happens elsewhere, it's still an interesting data point... – StilesCrisis Jan 07 '12 at 17:49
How do I get the address where the EXC_BAD_ACCESS occurs? – Dominik Seibold Jan 07 '12 at 19:16
I'm sorry, but I don't know which crash-log you are talking about. That's what I'm seeing, when the crash is happening: http://dominik.ws/screenshot.png – Dominik Seibold Jan 08 '12 at 01:52
You're running in the debugger, so no crash log is generated. – StilesCrisis Jan 08 '12 at 18:29
I'm a bit confused. If I don't run the app in the debugger, how can I see the NSLog-output? Anyway, I changed the debugger from GDB to LLDB and now I get assembler-code presented by Xcode, when the crash happens. The output of your NSLog-code is "Buffer at 0x05C26000" and the address, which seems to be responsible for the crash is 0x5C26001. Here is again a screenshot: http://dominik.ws/screenshot2.png – Dominik Seibold Jan 08 '12 at 22:42
From the screenshot I can see that the error is "EXC_ARM_DA_ALIGN", which means the code is crashing because of an unaligned write to memory. It looks like OpenGL is using opcodes that require proper alignment to execute properly. :( You can file a Radar but I think you might be stuck with a memcpy at this point, if OpenGL and AVAssetWriter both require aligned buffers. – StilesCrisis Jan 08 '12 at 23:16
I'm thinking your best bet is the fragment shader swizzle or some permutation thereof. You could probably do it in three fixed-function passes as well if you are tricky with vertex colors. – StilesCrisis Jan 09 '12 at 03:11

score 1 · Answer 3 · answered Jan 07 '12 at 18:12

Try calling

glPixelStorei(GL_PACK_ALIGNMENT,1)

before glReadPixels.

From the docs:

GL_PACK_ALIGNMENT

Specifies the alignment requirements for the start of each pixel row in memory. The allowable values are 1 (byte-alignment), 2 (rows aligned to even-numbered bytes), 4 (word-alignment), and 8 (rows start on double-word boundaries).

The default value is 4 (see glGet). This often gets mentioned as a troublemaker in various "OpenGL pitfalls" type lists, although this is generally more to do with its row padding effects than buffer alignment.

As an alternative approach, what happens if you malloc 4 extra bytes, do the glReadPixels as 4-byte aligned starting at buffer+4, and then pass your AVAssetWriter buffer+3 (although I've no idea whether AVAssetWriter is more tolerant of alignment issues) ?

glPixelStorei doesn't help. Your second solution works, but makes an additional memcpy-step necessary, which is not good for the performance. — Dominik Seibold, Jan 07 '12 at 19:21

hotpaw2 · Answer 4 · 2012-01-07T17:31:34.333

0

You will need to shift bytes by doing a memcpy or other copy operation. Modifying the pointers will leave them unaligned, which may or may not be within the capabilities of any underlying hardware (DMA bus widths, tile granularity, etc.)

edited Jan 07 '12 at 17:31

answered Jan 07 '12 at 17:18

hotpaw2

70,107
14
90
153

I don't see any reason why glReadPixels shouldn't be able to write into a pointer that's not aligned. It might not be optimally efficient, but it really shouldn't crash... – StilesCrisis Jan 07 '12 at 17:21

score -1 · Answer 5 · answered Jan 07 '12 at 17:21

-1

Using buffer+1 will mean the data is not written at the start of your malloc'd memory, but rather one byte in, so it will be writing over the end of your malloc'd memory, causing the crash.

If iOS's glReadPixels will only accept GL_RGBA then you'll have to go through and re-arrange them yourself I think.

UPDATE, sorry I missed the +1 in your malloc, StilesCrisis is probably right about the cause of the crash.

answered Jan 07 '12 at 17:21

DaedalusFall

8,335
6
30
43

He adds an extra byte onto his allocated buffer, though. – StilesCrisis Jan 07 '12 at 17:22

Converting RGBA to ARGB (glReadPixels -> AVAssetWriter)

5 Answers5

Linked