0

I'm using JavaCPP av_read_packet to read AVPackets from H264 (MPEG4 AVC) source, then I'm trying to pass only the video packets to Android's MediaCodec to decode them and to render into a Surface (for display). I've read it's necesary to use av_bitstream_filter_filter with h264_mp4toannexb filter when you use AVC source to be compatible with MediaCodec.

The problem is that dequeueOutputBuffer always return -1, and I don't know if the problem is when use the filter or any other reason.

Here is the code fragment:

bsfc = new AVBitStreamFilterContext();
bsfc = av_bitstream_filter_init("h264_mp4toannexb");

public void FilterPacket(AVPacket paquete){

    IntPointer size_pointer = new IntPointer(1);
    size_pointer.put(paquete.size());
    BytePointer data = new BytePointer();

    int sal = av_bitstream_filter_filter(bsfc, video_codec, (BytePointer)null, data, size_pointer, paquete.data(), paquete.size(), paquete.flags() & AV_PKT_FLAG_KEY );

    paquete.size(size_pointer.get());
    paquete.data(data);

}

And here is what's inside the decode loop:

public void DecoderLoop(AVPacket paquete){
    int inputBufferIndex = decoder.dequeueInputBuffer(5000);
    if (inputBufferIndex >= 0) {
        ByteBuffer bufferSalida = inputBuffers[inputBufferIndex]; 
        FilterPacket(paquete);
        bufferSalida.put(paquete.data().asByteBuffer());
        //double presentationTemp = (double)(paquete.pts()/paquete.duration()) * (1000000/30); 
        //long presentationTime = (long)presentationTemp;
        decoder.queueInputBuffer(inputBufferIndex, 0, paquete.size(), 0, 0);

    }

    int outputBufferIndex = decoder.dequeueOutputBuffer(info, 10000);
    if (outputBufferIndex >= 0) {
        // outputBuffer is ready to be processed or rendered.
        decoder.releaseOutputBuffer(outputBufferIndex, true); 
    } else if (outputBufferIndex == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) {
        outputBuffers = decoder.getOutputBuffers();
    } else if (outputBufferIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
        // Subsequent data will conform to new format.
        format = decoder.getOutputFormat();
    }
}

dequeueInputBuffer returns a positive value, and the av_bitstream_filter_filter returns 1 (but i don't know if it's working), but dequeueOutputBuffers return -1 always. Can anyone help me where can be the problem?

I suspect that the problem can be in one of these:

  • av_bitstream_filter_filter

  • bufferSalida.put(paquete.data().asByteBuffer()); ¿Can i use these way?

  • If I use .asByteBuffer, can i use paquete.size() or may I use something different?

Thank you very much

WotG
  • 5
  • 2
  • Perhaps http://bigflake.com/mediacodec/#q3 ? – fadden Oct 07 '14 at 15:16
  • In Q3 they configure the decoder via MediaExtractor, but in this case (no Media Extractor) i configure it like this: ` type = "video/avc"; decoder = MediaCodec.createDecoderByType(type); ... format = MediaFormat.createVideoFormat(type, codec_input.width(), codec_input.height()); decoder.configure(format, surface, null, 0); decoder.start(); inputBuffers = decoder.getInputBuffers(); outputBuffers = decoder.getOutputBuffers();` – WotG Oct 07 '14 at 16:03
  • And i haven't BUFFER_FLAG_CODEC_CONFIG because i'm not using MediaDecoder. The packets come from AVPackets from javacpp – WotG Oct 07 '14 at 16:11
  • The point of Q3 is that the MediaCodec decoder needs to have the SPS/PPS values passed in via a special packet or a buffer associated with the MediaFormat. Without it the decoder won't start processing data. – fadden Oct 08 '14 at 01:02
  • Ok, now I understand the point, and I'm pretty sure you are right. Thank you very much – WotG Oct 08 '14 at 13:08

1 Answers1

1

I'm not familiar with this JavaCPP wrapping of the libavcodec libraries, but in addition to the plain AVPackets, you need AVCodecContext.extradata as well (after initializing the bitstream filter, and after filtering the first packet). You need to pass the data from extradata either as a ByteBuffer with the name "csd-0" in the input MediaFormat that you pass to configure, or as a separate buffer with BUFFER_FLAG_CODEC_CONFIG before sending the first AVPacket.

mstorsjo
  • 12,983
  • 2
  • 39
  • 62
  • fadden and you drive me to the same (and correct) way. I need to configure correctly the decoder before sending the packets. Now I don't know how to pass that information to the MediaFormat (before calling configure() method), and what are the minimun parameters I need to get it working. But lets try. Thank you very much – WotG Oct 08 '14 at 13:18