6

I am saving frames to H264 format and I saved audio to aac format. Then I concatenate of these format to creating mp4 format using ffmpeg player in android but When I concatenate Audio and Video, audio getting back of video they aren't playing at synchronous mode,how can I played video and audio at syncronous mode? H264 video format is 6 second, audio format is 8 second When I concatenate of these getting 8 second and audio coming longer and occuring asyncronous.

Recording Audio to AAC format

    recorder = new MediaRecorder();

    recorder.setAudioSource(MediaRecorder.AudioSource.MIC);
    recorder.setOutputFormat(MediaRecorder.OutputFormat.MPEG_4); 
    recorder.setAudioEncoder(MediaRecorder.AudioEncoder.AAC);    
    recorder.setAudioEncodingBitRate(48000);//48000
    recorder.setAudioSamplingRate(720);//16000
    recorder.setOutputFile(path2);
    try {
        recorder.prepare();
    } catch (IOException e) {
        e.printStackTrace();
    }
    recorder.start();

Saving Video to H264 format

        MediaFormat mediaFormat = MediaFormat.createVideoFormat("video/avc",
                1280,
                720);
        mediaFormat.setInteger(MediaFormat.KEY_COLOR_FORMAT,
                MediaCodecInfo.CodecCapabilities.COLOR_FormatYUV420Planar);
        mediaFormat.setInteger(MediaFormat.KEY_BIT_RATE, 6000000);
        mediaFormat.setInteger(MediaFormat.KEY_FRAME_RATE, 720); //video second

        mediaFormat.setInteger(MediaFormat.KEY_I_FRAME_INTERVAL, 5);
        try {
            mMediaCodec = MediaCodec.createEncoderByType("video/avc");
        } catch (IOException e) {
            e.printStackTrace();
        }
        mMediaCodec.configure(mediaFormat,
                null,
                null,
                MediaCodec.CONFIGURE_FLAG_ENCODE);

        mMediaCodec.start();

//Video format H264
private synchronized void encode(byte[] data) {

    ByteBuffer[]  inputBuffers = mMediaCodec.getInputBuffers();
    ByteBuffer[]  outputBuffers = mMediaCodec.getOutputBuffers();

    int inputBufferIndex = mMediaCodec.dequeueInputBuffer(-1);
    if (inputBufferIndex >= 0) {
        ByteBuffer inputBuffer = inputBuffers[inputBufferIndex];
        inputBuffer.capacity();
        inputBuffer.clear();
        inputBuffer.put(data);
        mMediaCodec.queueInputBuffer(inputBufferIndex, 0, data.length, 0, 0);

    } else {
        return;
    }

    MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo();
    int outputBufferIndex = mMediaCodec.dequeueOutputBuffer(bufferInfo, 0);
    Log.i(TAG, "outputBufferIndex-->" + outputBufferIndex);
    do {
        if (outputBufferIndex >= 0) {
            ByteBuffer outBuffer = outputBuffers[outputBufferIndex];
            System.out.println("buffer info-->" + bufferInfo.offset + "--"
                    + bufferInfo.size + "--" + bufferInfo.flags + "--"
                    + bufferInfo.presentationTimeUs);
            byte[] outData = new byte[bufferInfo.size];
            outBuffer.get(outData);
            try {
                if (bufferInfo.offset != 0) {
                    fos.write(outData, bufferInfo.offset, outData.length
                            - bufferInfo.offset);
                } else {
                    fos.write(outData, 0, outData.length);
                }
                fos.flush();
                Log.i(TAG, "out data -- > " + outData.length);
                mMediaCodec.releaseOutputBuffer(outputBufferIndex, false);
                outputBufferIndex = mMediaCodec.dequeueOutputBuffer(bufferInfo,
                        0);
            } catch (IOException e) {
                e.printStackTrace();
            }
        } else if (outputBufferIndex == MediaCodec.INFO_OUTPUT_BUFFERS_CHANGED) {
            outputBuffers = mMediaCodec.getOutputBuffers();
        } else if (outputBufferIndex == MediaCodec.INFO_OUTPUT_FORMAT_CHANGED) {
            MediaFormat format = mMediaCodec.getOutputFormat();
        }
    } while (outputBufferIndex >= 0);
}

Concatenate Video and Audio with using ffmpeg

String[] cmd = {"-i", h264_video_path, "-i", aac_audio_path, "-c", "copy", "-map","0:v:0","-map","1:a:0", outpath_mp4};

    try {
                    //FFMPEG execute command
                    executeCommand(cmd);
                } catch (FFmpegCommandAlreadyRunningException e) {
                    e.printStackTrace();
                }
Diego
  • 937
  • 8
  • 24
  • Are you saving the H264 stream as raw stream? You only show the encoding part. If this is the case, the H264 stream does not contain any presentation timestamp and the video might be running faster than it should. Also I'd suggest that you use MediaCodec and MediaMuxer to join both streams at runtime if that is an option for you. – ChrisBe Jan 07 '19 at 11:10
  • thank you @ChrisBe ,I added encode(data) method for saving H264,how to I use MediaCodec and MediaMuxer at each time for the syncronazation.Can you give me a example? – Diego Jan 07 '19 at 12:05
  • I used MediaCodec for saving H264 format but I didn't use MediaMuxer ,I used ffmpeg for the join H264 and audio – Diego Jan 07 '19 at 12:09
  • Just to make sure I get the scenario: You get raw frames from an external camera, encode it to H264 frames and write them to a file sequentially. Audio is recorded by device microphone, encoded into AAC and stored as MP4. – ChrisBe Jan 07 '19 at 16:16
  • If the extra step of concatenating two files doesn't bother you, you would just change the saving method to use MediaMuxer with one track and calculate the presentation timestap yourself. An basic example of how to use MediaMuxer can be found [here](https://developer.android.com/reference/android/media/MediaMuxer) – ChrisBe Jan 07 '19 at 16:22

0 Answers0