2

I try to render a raw h264 video to a surface (after decoding) and write it at the same time to a file. The rendering is working fine but when I want to get the current output buffer, it always have a size of 8 and the output file have a size of 3,87 Ko.

It seems like the output buffer is locked by the surface (ANativeWindow)? Anyone can give me an advice to do it without creating another codec?

The codec is configured with an output surface :

 if (AMEDIA_OK == AMediaCodec_configure(d->codec, d->format, d->window /*the native window */, NULL, 0)

Here is the code snipet when I try to get the output buffer :

if (!d->sawOutputEOS) {
    AMediaCodecBufferInfo info;
    auto status = AMediaCodec_dequeueOutputBuffer(d->codec, &info, -1);
    if (status >= 0) {
        if (info.flags & AMEDIACODEC_BUFFER_FLAG_END_OF_STREAM) {
            LOGV("output EOS");
            d->sawOutputEOS = true;
            d->isPlaying = false;
        }
        int64_t delay = 333000;
        usleep((useconds_t )delay / 15);
        size_t size;
        // here i get the output buffer
        uint8_t *outputbuffer = AMediaCodec_getOutputBuffer(d->codec,status,&size);
        write(d->fd1,outputbuffer,size); // the output is always 0
        LOGV("%d",size); // the size is always 8
        LOGV("FRAME num : %d", counter[d->nb]++);
        AMediaCodec_releaseOutputBuffer(d->codec, status, info.size != 0);
        if (d->renderonce) {
            d->renderonce = false;
            return;
        }
    } else if (status == AMEDIACODEC_INFO_OUTPUT_BUFFERS_CHANGED) {
        LOGV("output buffers changed");
    } else if (status == AMEDIACODEC_INFO_OUTPUT_FORMAT_CHANGED) {
        auto format = AMediaCodec_getOutputFormat(d->codec);
        LOGV("format changed to: %s", AMediaFormat_toString(format));
        AMediaFormat_delete(format);
        d->formatChanged = true;
    } else if (status == AMEDIACODEC_INFO_TRY_AGAIN_LATER) {
        LOGV("no output buffer right now");
    } else {
        LOGV("unexpected info code: %zd", status);
    }
}

Thanks in advance

E.Abdel
  • 1,992
  • 1
  • 13
  • 24

1 Answers1

2

It's not locked; you asked the decoder to work for display, so it used the fastest route to display, without exposing the pixels to readable memory. You may find that the format is the opaque COLOR_FormatSurface, as explained here.

Alex Cohn
  • 56,089
  • 9
  • 113
  • 307
  • Hi @Alex, and thanks for your answer. So if I understand the only way to render and write frame at the same time is to configure the decoder without an output surface and then get the output buffer, write it to the output file and surface buffer at the same time? – E.Abdel Apr 23 '18 at 11:18
  • More or less that's the way, but the question is whether the result will be fast enough for live view. If it isn't, you can try to configure the decoder to *both* surface and to memory. In the worst case, run two decoders in separate threads. The thread that writes to file may lag behind, but usually this does not matter. Another alternative is to read pixels from the surface directly. This may be slower, but has a benefit: you don't need to convert YUV to RGB. – Alex Cohn Apr 23 '18 at 11:44
  • I confirm, I am trying it now and it is very bad for live view, especially that in my case the decoder output color is Semi-planar, so i have to swap UV...UV... to UU...VV... before converting it to RGB I will try your suggestions to see wich is the better, thanks again – E.Abdel Apr 23 '18 at 11:53
  • Well, actually to display NV21 frames you don't need to swap chroma and convert to RGB yourself, see https://stackoverflow.com/a/49980557/192373 – Alex Cohn Apr 23 '18 at 11:58
  • 1
    I will check that, I did it when I had to save frames to an output file. Just a last question, can we configure mediacodec twice? thanks – E.Abdel Apr 23 '18 at 12:02
  • 1
    MediaCodec is not a singleton. You can [AMediaCodec_createCodecByName()](https://developer.android.com/ndk/reference/ndk_media_codec_8h.html#a678fff1971bc62e32140e12f11a77044) twice or even more. But this resource is limited, so don't keep unnecessary instances. – Alex Cohn Apr 23 '18 at 12:32