10

I'm trying to record Unity game audio to mp4 file by using OnAudioFilterRead function.

function OnAudioFilterRead(var data:float[], var channels:int)

Overview of my solution.
Step1. Convert data samples that get from Unity to bytes[]

private void Convert2Bytes(float[] samples) {
    Int16[] intData = new Int16[samples.Length];
    //converting in 2 float[] steps to Int16[], //then Int16[] to Byte[]

    Byte[] bytesData = new Byte[samples.Length * 2];
    //bytesData array is twice the size of
    //dataSource array because a float converted in Int16 is 2 bytes.

    float rescaleFactor = 32767; //to convert float to Int16

    for (int i = 0; i<samples.Length; i++) {
        intData[i] = (short) ((samples[i] * gain) * rescaleFactor);
        int n =  i*2;
        bytesData[n] = (byte)(intData[i]& 0x00FF);
        bytesData[n+1] = (byte)(((intData[i]& 0xFF00) >> 8) );
        if (muteSceneAudio)
            samples[i] = 0.0f;
    }
    return bytesData;
}

Step2. Push bytes array to MediaCoder mAudioEncoder

ByteBuffer[] inputBuffers = mAudioEncoder.getInputBuffers();
if (inputBufferIndex >= 0) {
    ByteBuffer inputBuffer = inputBuffers[inputBufferIndex];
    inputBuffer.clear();
    inputBuffer.put(data, 0, countOfByte);
    int inputLength = countOfByte;
    presentationTimeNs -= (inputLength / SAMPLE_RATE) / 1000000000;
    if (inputLength == AudioRecord.ERROR_INVALID_OPERATION)
        Log.printLog(Log.ERROR, "Audio read error");

    long presentationTimeUs = (presentationTimeNs - startWhen) / 1000;
    Log.printLog(Log.INFO, "queueing " + inputLength + " audio bytes with pts "
            + presentationTimeUs);
    if (endOfStream) {
        Log.printLog(Log.INFO, "EOS received in sendAudioToEncoder");
        mAudioEncoder.queueInputBuffer(inputBufferIndex, 0,
                inputLength, presentationTimeUs,
                MediaCodec.BUFFER_FLAG_END_OF_STREAM);
        eosSentToAudioEncoder = true;
    } else {
        mAudioEncoder.queueInputBuffer(inputBufferIndex, 0,
                inputLength, presentationTimeUs, 0);
    }
}

Step3. DequeueOutputBuffer and write to video

int encoderStatus = encoder.dequeueOutputBuffer(bufferInfo, TIMEOUT_USEC);
if (encoderStatus == MediaCodec.INFO_TRY_AGAIN_LATER) {
// Bug here
}

Error log (it happens when stop recording):

AndroidJavaException: java.lang.IllegalStateException
    at UnityEngine.AndroidJNISafe.CheckException () [0x00000] in <filename unknown>:0 
    at UnityEngine.AndroidJNISafe.CallVoidMethod (IntPtr obj, IntPtr methodID, UnityEngine.jvalue[] args) [0x00000] in <filename unknown>:0 
    at UnityEngine.AndroidJavaObject._Call (System.String methodName, System.Object[] args) [0x00000] in <filename unknown>:0 
    at UnityEngine.AndroidJavaObject.Call (System.String methodName, System.Object[] args) [0x00000] in <filename unknown>:0 
    at VideoCapture.stopAudioRecord () [0x00000] in <filename unknown>:0 
    at OOTAudioRecord.SaveStreamStop () [0x00000] in <filename unknown>:0 

When I tried to record audio from mic, it worked Any solution for my issue?

Maifee Ul Asad
  • 3,992
  • 6
  • 38
  • 86
huythang
  • 313
  • 1
  • 3
  • 13
  • private void Convert2Bytes(float[] samples) is returning Byte[], but it's declared return type is void. – Chris H Apr 25 '18 at 18:16

0 Answers0