Presentation time of Audio buffer and Video buffer are not equal

Question

I'm trying to create an app that does live Video & Audio recording, using AVFoundation. Also using AVAssetWriter I'm writing the buffers to a local file.

For the Video CMSampleBuffer I'm Using the AVCaptureVideoDataOutputSampleBufferDelegate output in AVCaptureSession which is straightforward.

For the Audio CMSampleBuffer I'm creating the buffer from the AudioUnit record callback. The way I'm calculating the presentation time for the Audio buffer is like so:

        var timebaseInfo = mach_timebase_info_data_t(numer: 0, denom: 0)
        let timebaseStatus = mach_timebase_info(&timebaseInfo)
        if timebaseStatus != KERN_SUCCESS {
            debugPrint("not working")
            return
        }
        let hostTime = time * UInt64(timebaseInfo.numer / timebaseInfo.denom)
        let presentationTIme = CMTime(value: CMTimeValue(hostTime), timescale: 1000000000)
        let duration = CMTime(value: CMTimeValue(1), timescale: CMTimeScale(self.sampleRate))

        var timing = CMSampleTimingInfo(
            duration: duration,
            presentationTimeStamp: presentationTIme,
            decodeTimeStamp: CMTime.invalid
        )

self.sampleRate is a variable that changes at the recording start, but most of the times is 48000.

When getting the CMSampleBuffers of both the video and the audio the presentation times has a really big difference.

Audio - CMTime(value: 981750843366125, timescale: 1000000000, flags: __C.CMTimeFlags(rawValue: 1), epoch: 0)

Video - CMTime(value: 997714237615541, timescale: 1000000000, flags: __C.CMTimeFlags(rawValue: 1), epoch: 0)

This creates a big gap when trying to write the buffers to the file.

My questions is

Am I calculating the presentation Time of the audio buffer correctly? if so, what am I missing?
How can I make sure the Audio & the Video are in the same area of time (I know that there should be a small millisecond difference between them)

Aren't you getting truncation with `time * UInt64(timebaseInfo.numer / timebaseInfo.denom)`? Shouldn't that be `(time*numer)/denom`? — Rhythmic Fistman, Jun 23 '21 at 16:17

score 0 · Answer 1 · answered Jun 24 '21 at 08:59

Ok, so this was my fault. As Rhythmic Fistman in the comments suggested, I was getting truncation with my time calculation:

let hostTime = time * UInt64(timebaseInfo.numer / timebaseInfo.denom)

Changing to this calculation fixed it

let hostTime = (time * UInt64(timebaseInfo.numer)) / UInt64(timebaseInfo.denom)

Presentation time of Audio buffer and Video buffer are not equal

1 Answers1