Android MediaCodec AMR-NB realtime decoding

Question

I'm working on VoIP via a socket like API. (With a narrow band connection)

I need to encode each voice frame (20ms) and send via said api, then decode it on the other side.

I tried working with Opus through NDK but it wasn't going anywhere so I decided using MediaCodec to encode/decode with AMR-NB.

The Encoding seems to work, results in a buffer of the expected size (160 raw bytes into 20 encoded bytes + header at 7.9Kbps)

But when I handle the encoded buffer and try to decode it I recieve a INFO_OUTPUT_FORMAT_CHANGED result.

Encoder:

// Set up recorder
    int recordBufferSize = AudioRecord.getMinBufferSize(SAMPLE_RATE, AudioFormat.CHANNEL_CONFIGURATION_MONO, AudioFormat.ENCODING_PCM_16BIT);
    arec = new AudioRecord(MediaRecorder.AudioSource.MIC, SAMPLE_RATE, AudioFormat.CHANNEL_CONFIGURATION_MONO, AudioFormat.ENCODING_PCM_16BIT, recordBufferSize);

    // Set Up codec
    try {
        encoder = MediaCodec.createEncoderByType(MediaFormat.MIMETYPE_AUDIO_AMR_NB);
        MediaFormat format = new MediaFormat();
        format.setString(MediaFormat.KEY_MIME, MediaFormat.MIMETYPE_AUDIO_AMR_NB);
        format.setInteger(MediaFormat.KEY_SAMPLE_RATE, SAMPLE_RATE);
        format.setInteger(MediaFormat.KEY_CHANNEL_COUNT, 1);
        format.setInteger(MediaFormat.KEY_BIT_RATE, bitrate.getVal());
        encoder.configure(format, null, null, MediaCodec.CONFIGURE_FLAG_ENCODE);
    } catch (IOException e) {
        e.printStackTrace();
        return;
    }

    // Start Recording
    int read;
    byte[] buffer1 = new byte[bufferSize];

    ByteBuffer[] inputBuffers;
    ByteBuffer[] outputBuffers;

    ByteBuffer inputBuffer;
    ByteBuffer outputBuffer;

    MediaCodec.BufferInfo bufferInfo;
    int inputBufferIndex;
    int outputBufferIndex;

    byte[] outData;

    Frame frame;
    try {
        encoder.start();
        arec.startRecording();
        isRecording = true;
        while (isRecording) {
            read = arec.read(buffer1, 0, bufferSize);

            inputBuffers = encoder.getInputBuffers();
            outputBuffers = encoder.getOutputBuffers();
            inputBufferIndex = encoder.dequeueInputBuffer(-1);
            if (inputBufferIndex >= 0) {
                inputBuffer = inputBuffers[inputBufferIndex];
                inputBuffer.clear();

                inputBuffer.put(buffer1);

                encoder.queueInputBuffer(inputBufferIndex, 0, buffer1.length, 0, 0);
            }

            bufferInfo = new MediaCodec.BufferInfo();
            outputBufferIndex = encoder.dequeueOutputBuffer(bufferInfo, 0);


            while (outputBufferIndex >= 0) {
                outputBuffer = outputBuffers[outputBufferIndex];

                outputBuffer.position(bufferInfo.offset);
                outputBuffer.limit(bufferInfo.offset + bufferInfo.size);

                outData = new byte[bufferInfo.size];
                outputBuffer.get(outData);


                //-------------
                frame = new Frame(outData);
                handler.onFrameEncoded(frame);
                //------------

                encoder.releaseOutputBuffer(outputBufferIndex, false);
                outputBufferIndex = encoder.dequeueOutputBuffer(bufferInfo, 0);

            }
        }
        encoder.stop();
        arec.stop();
    } catch (Exception e) {
        e.printStackTrace();
    }

And the Decoder:

@Override
        public void onFrameEncoded(Frame frame) {
            try {
                MediaCodec decoder = MediaCodec.createDecoderByType(MediaFormat.MIMETYPE_AUDIO_AMR_NB);
                MediaFormat format = new MediaFormat();
                format.setString(MediaFormat.KEY_MIME, MediaFormat.MIMETYPE_AUDIO_AMR_NB);
                format.setInteger(MediaFormat.KEY_SAMPLE_RATE, SAMPLE_RATE);
                format.setInteger(MediaFormat.KEY_CHANNEL_COUNT, 1);
                format.setInteger(MediaFormat.KEY_BIT_RATE, 7950);
                decoder.configure(format, null, null, 0);
                decoder.start();

                byte[] outData;

                ByteBuffer[] inputBuffers;
                ByteBuffer[] outputBuffers;

                ByteBuffer inputBuffer;
                ByteBuffer outputBuffer;

                MediaCodec.BufferInfo bufferInfo;
                int inputBufferIndex;
                int outputBufferIndex;
                inputBuffers = decoder.getInputBuffers();
                outputBuffers = decoder.getOutputBuffers();


                // Fill decoder input buffer
                inputBufferIndex = decoder.dequeueInputBuffer(-1);
                if (inputBufferIndex >= 0) {
                    inputBuffer = inputBuffers[inputBufferIndex];
                    inputBuffer.clear();

                    inputBuffer.put(frame.Buffer);

                    decoder.queueInputBuffer(inputBufferIndex, 0, frame.Buffer.length, 0, 0);
                }


                // Get Output
                bufferInfo = new MediaCodec.BufferInfo();
                outputBufferIndex = decoder.dequeueOutputBuffer(bufferInfo, -1);
                if (outputBufferIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) { // Just a check I threw in
                    MediaFormat format2 = decoder.getOutputFormat(); // Returns format RAW
                }

                while (outputBufferIndex >= 0) {
                    outputBuffer = outputBuffers[outputBufferIndex];

                    outputBuffer.position(bufferInfo.offset);
                    outputBuffer.limit(bufferInfo.offset + bufferInfo.size);

                    outData = new byte[bufferInfo.size];
                    outputBuffer.get(outData);


                    // Log.d("Decoder", outData.length + " bytes encoded");

                    decoder.releaseOutputBuffer(outputBufferIndex, false);
                    outputBufferIndex = decoder.dequeueOutputBuffer(bufferInfo, 0);
                }
            } catch (IOException e) {
                e.printStackTrace();
            }
        }

I've searched everywhere for INFO_OUTPUT_FORMAT_CHANGED but found nothing that could help me..

score 1 · Accepted Answer · answered Aug 11 '15 at 15:02

1

That's the expected behavior. If you were using MediaMuxer, you would pass that MediaFormat to addTrack(), which needs some bits of information that have to come from MediaCodec. This behavior was added in API 18 when MediaMuxer was introduced.

See EncodeAndMuxTest for an example.

If you're not using MediaMuxer, you can ignore it. (You may want to log it just to confirm that everything is still as it should be.)

answered Aug 11 '15 at 15:02

fadden

51,356
5
116
166

Problem is I'm not using MediaMuxer, as I'm not outputing to file, instead I need to decode the frame and feed it to the AudioTrack. I can't ignore it because I'm supposed to recieve the output buffer index and I recieve `INFO_OUTPUT_FORMAT_CHANGED` instead. with an empty BufferInfo. I need a way to decode data frame by frame. – Eitan F Aug 12 '15 at 06:44
The "output format changed" result is not *instead* of data, it's *in addition* to data, right before the data appears. If you're not seeing any data come out after that, then something is wrong. Your code seems to be seeing the one result and then bailing. It also seems to be expecting to be able to submit one buffer and immediately receive one buffer, which may be unwise -- that won't work for video codecs, which tend to want a few packets in queue, though it might work for audio. See the bigflake site for info and examples. – fadden Aug 12 '15 at 15:17

Android MediaCodec AMR-NB realtime decoding

1 Answers1