Trying to append CVPixelBuffers to AVAssetWriterInputPixelBufferAdaptor at the intended framerate

Question

I'm trying to append CVPixelBuffers to AVAssetWriterInputPixelBufferAdaptor at the intended framerate, but it seems to be too fast, and my math is off. This isn't capturing from the camera, but capturing changing images. The actual video is much to fast than the elapsed time it was captured.

I have a function that appends the CVPixelBuffer every 1/24 of a second. So I'm trying to add an offset of 1/24 of a second to the last time.

I've tried:

let sampleTimeOffset = CMTimeMake(value: 100, timescale: 2400)

and:

let sampleTimeOffset = CMTimeMake(value: 24, timescale: 600)

and:

let sampleTimeOffset = CMTimeMakeWithSeconds(0.0416666666, preferredTimescale: 1000000000)

I'm adding onto the currentSampleTime and appending like so:

self.currentSampleTime = CMTimeAdd(currentSampleTime, sampleTimeOffset)

let success = self.assetWriterPixelBufferInput?.append(cv, withPresentationTime: currentSampleTime)

One other solution I thought of is get the difference between the last time and the current time, and add that onto the currentSampleTime for accuracy, but unsure how to do it.

score 1 · Answer 1 · answered Nov 07 '19 at 22:35

I found a way to accurately capture the time delay by comparing the last time in milliseconds compared to the current time in milliseconds.

First, I have a general current milliseconds time function:

func currentTimeInMilliSeconds()-> Int
{
    let currentDate = Date()
    let since1970 = currentDate.timeIntervalSince1970
    return Int(since1970 * 1000)
}

When I create a writer, (when I start recording video) I set a variable in my class to the current time in milliseconds:

currentCaptureMillisecondsTime = currentTimeInMilliSeconds()

Then in my function that's supposed to be called 1/24 of a second is not always accurate, so I need to get the difference in milliseconds between when I started writing, or my last function call.

Do a conversion of milliseconds to seconds, and set that to CMTimeMakeWithSeconds.

let lastTimeMilliseconds = self.currentCaptureMillisecondsTime
let nowTimeMilliseconds = currentTimeInMilliSeconds()
let millisecondsDifference = nowTimeMilliseconds - lastTimeMilliseconds

// set new current time
self.currentCaptureMillisecondsTime = nowTimeMilliseconds

let millisecondsToSeconds:Float64 = Double(millisecondsDifference) * 0.001

let sampleTimeOffset = CMTimeMakeWithSeconds(millisecondsToSeconds, preferredTimescale: 1000000000)

I can now append my frame with the accurate delay that actually occurred.

self.currentSampleTime = CMTimeAdd(currentSampleTime, sampleTimeOffset)

let success = self.assetWriterPixelBufferInput?.append(cv, withPresentationTime: currentSampleTime)

When I finish writing the video and I save it to my camera roll, it is the exact duration from when I was recording.

Yo, this is great. You have a complete code example somewhere? Having a hard time with the currentSampleTime variable. Not very clear. — Jon Vogel, May 16 '20 at 00:55
This is very old to me, so I'm fuzzy on the details of how this worked now too. currentSampleTime is CMTime. It can be defined as var currentSampleTime: CMTime? Hope this helps. — Chewie The Chorkie, May 16 '20 at 17:40
And it gets reset on every frame pass by initialization with its self as a value and the computed sample time offset? — Jon Vogel, May 16 '20 at 21:26
What do you seed it with if its definition is null? CMTime.zero or CMTimeMakeWithSeconds(self.currentTimeInMilliSeconds(), preferredTimescale: 1000000000) — Jon Vogel, May 16 '20 at 21:29
Finally, what did you seed the AVAssetWriter startSession(atSourceTime: ?) with? CMTime.zero or the current time? — Jon Vogel, May 16 '20 at 21:34

Trying to append CVPixelBuffers to AVAssetWriterInputPixelBufferAdaptor at the intended framerate

1 Answers1