Let me start by saying I have searched for an answer to this question for a long time and have read through many SO posts, but none have provided the answer I need
I am attempting to record multiple video segments using multiple AVAssetWriters/AVAssetWriterInputs, which I instantiate and queue up immediately.
The problem I am facing is that while the videos are recorded properly, when they enter an AVQueuePlayer to be played back in sequential order, there is a random occurrence by which one of the videos will freeze but continue to play audio.
Preparing the AVAssetWriters/VideoInputs/AudioInputs:
private func setupAssetWriters() {
guard assetWriterOne == nil else { return }
// This just returns a URL with ".mov" as the file suffix in the absoluteString.
outputFileURLOne = createRandomizedURL(withMediaFileType: .mov)
(assetWriterOne, videoAssetWriterInputOne, audioAssetWriterInputOne) = configureAssetWriter(withInput: outputFileURLOne)
guard assetWriterTwo == nil else { return }
outputFileURLTwo = createRandomizedURL(withMediaFileType: .mov)
(assetWriterTwo, videoAssetWriterInputTwo, audioAssetWriterInputTwo) = configureAssetWriter(withInput: outputFileURLTwo)
guard assetWriterThree == nil else { return }
outputFileURLThree = createRandomizedURL(withMediaFileType: .mov)
(assetWriterThree, videoAssetWriterInputThree, audioAssetWriterInputThree) = configureAssetWriter(withInput: outputFileURLThree)
guard assetWriterFour == nil else { return }
outputFileURLFour = createRandomizedURL(withMediaFileType: .mov)
(assetWriterFour, videoAssetWriterInputFour, audioAssetWriterInputFour) = configureAssetWriter(withInput: outputFileURLFour)
}
private func configureAssetWriter(withInput url: URL) -> (assetWriter: AVAssetWriter?, videoAssetWriterInput: AVAssetWriterInput?, audioAssetWriterInput: AVAssetWriterInput?){
do {
let recommendedVideoSettings: [String: Any]? = videoDataOutput?.recommendedVideoSettingsForAssetWriter(writingTo: AVFileType.mov)
let assetWriter = try VideoAssetWriter(url: url, fileType: AVFileType.mov)
guard
assetWriter.canApply(outputSettings: recommendedVideoSettings, forMediaType: AVMediaType.video),
assetWriter.canApply(outputSettings: audioSettings, forMediaType: AVMediaType.audio)
else { return (nil, nil, nil) }
let videoAssetWriterInput = AVAssetWriterInput(mediaType: AVMediaType.video, outputSettings: recommendedVideoSettings, sourceFormatHint: videoFormatDescription)
let audioAssetWriterInput = AVAssetWriterInput(mediaType: AVMediaType.audio, outputSettings: audioSettings)
guard
assetWriter.canAdd(videoAssetWriterInput),
assetWriter.canAdd(audioAssetWriterInput)
else { return (nil, nil, nil) }
assetWriter.add(videoAssetWriterInput)
assetWriter.add(audioAssetWriterInput)
videoAssetWriterInput.expectsMediaDataInRealTime = true
audioAssetWriterInput.expectsMediaDataInRealTime = true
return (assetWriter, videoAssetWriterInput, audioAssetWriterInput)
} catch let error {
print("error getting AVAssetWriter: \(error.localizedDescription)")
return (nil, nil, nil)
}
}
Processing the CMSampleBuffers
Now that the AVAssetWriters and their inputs are initialized, I press a record button which changes a Bool value to let the AVCaptureVideoDataOutputSampleBufferDelegate know to begin capturing CMSampleBuffers:
func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {
if isRecording {
processVideoSamples(withOutput: output, sampleBuffer: sampleBuffer)
}
}
private func processVideoSamples(withOutput output: AVCaptureOutput, sampleBuffer: CMSampleBuffer) {
switch currentVideoInProcess {
case 1:
processSamples(usingAssetWriter: assetWriterOne, videoAssetWriterInput: videoAssetWriterInputOne, audioAssetWriterInput: audioAssetWriterInputOne, withOutput: output, sampleBuffer: sampleBuffer)
break
case 2:
processSamples(usingAssetWriter: assetWriterTwo, videoAssetWriterInput: videoAssetWriterInputTwo, audioAssetWriterInput: audioAssetWriterInputTwo, withOutput: output, sampleBuffer: sampleBuffer)
break
case 3:
processSamples(usingAssetWriter: assetWriterThree, videoAssetWriterInput: videoAssetWriterInputThree, audioAssetWriterInput: audioAssetWriterInputThree, withOutput: output, sampleBuffer: sampleBuffer)
break
case 4:
processSamples(usingAssetWriter: assetWriterFour, videoAssetWriterInput: videoAssetWriterInputFour, audioAssetWriterInput: audioAssetWriterInputFour, withOutput: output, sampleBuffer: sampleBuffer)
break
default:
break
}
}
private func processSamples(usingAssetWriter assetWriter: VideoAssetWriter?, videoAssetWriterInput: AVAssetWriterInput?, audioAssetWriterInput: AVAssetWriterInput?, withOutput output: AVCaptureOutput, sampleBuffer: CMSampleBuffer) {
guard CMSampleBufferDataIsReady(sampleBuffer) else { return }
guard
assetWriter != nil,
videoAssetWriterInput != nil,
audioAssetWriterInput != nil
else { return }
if assetWriter?.status == .unknown {
if let _ = output as? AVCaptureVideoDataOutput {
print("\n STARTED RECORDING")
assetWriter?.startWriting()
closingTime = sampleTime
let startRecordingTime = CMSampleBufferGetPresentationTimeStamp(sampleBuffer)
assetWriter?.startSession(atSourceTime: startRecordingTime)
} else {
print("output type unknown")
return
}
}
if assetWriter?.status == .failed { return }
guard assetWriter?.status == .writing else { return }
if let _ = output as? AVCaptureVideoDataOutput {
if (videoAssetWriterInput?.isReadyForMoreMediaData)! {
videoAssetWriterInput?.append(sampleBuffer)
if assetWriter?.hasWrittenFirstVideoSample == false {
print("added 1st video frame")
assetWriter?.hasWrittenFirstVideoSample = true
}
} else {
print("video writer not ready")
}
} else if let _ = output as? AVCaptureAudioDataOutput {
if (audioAssetWriterInput?.isReadyForMoreMediaData)! && assetWriter?.hasWrittenFirstVideoSample == true {
audioAssetWriterInput?.append(sampleBuffer)
} else {
print("audio writer not ready OR video not written yet")
}
}
}
Getting the video segments from each AVAssetWriter
As for getting the video segments, I call this endRecording function at specified intervals in order to get a video from the outputURL of each AVAssetWriter:
private func endRecordingOfCurrentWriter(completion: @escaping (_ fileURL: URL?, _ error: Error?) -> ()) {
// currentVideoInProcess is just an Int to keep track of which AVAssetWriter is current processing CMSampleBuffers
switch currentVideoInProcess {
case 1:
endRecording(atOutputURL: outputFileURLOne, usingAssetWriter: assetWriterOne, videoAssetWriterInput: videoAssetWriterInputOne, audioAssetWriterInput: audioAssetWriterInputOne) { (fileURL, error) in
completion(fileURL, error)
}
break
case 2:
endRecording(atOutputURL: outputFileURLTwo, usingAssetWriter: assetWriterTwo, videoAssetWriterInput: videoAssetWriterInputTwo, audioAssetWriterInput: audioAssetWriterInputTwo) { (fileURL, error) in
completion(fileURL, error)
}
break
case 3:
endRecording(atOutputURL: outputFileURLThree, usingAssetWriter: assetWriterThree, videoAssetWriterInput: videoAssetWriterInputThree, audioAssetWriterInput: audioAssetWriterInputThree) { (fileURL, error) in
completion(fileURL, error)
}
break
case 4:
endRecording(atOutputURL: outputFileURLFour, usingAssetWriter: assetWriterFour, videoAssetWriterInput: videoAssetWriterInputFour, audioAssetWriterInput: audioAssetWriterInputFour) { (fileURL, error) in
completion(fileURL, error)
}
break
default: break
}
currentVideoInProcess += 1
}
private func endRecording(atOutputURL outputURL: URL, usingAssetWriter assetWriter: VideoAssetWriter?, videoAssetWriterInput: AVAssetWriterInput?, audioAssetWriterInput: AVAssetWriterInput?, completion: @escaping (_ fileURL: URL?, _ error: Error?) -> ()) {
if assetWriter?.status.rawValue == 1 {
videoAssetWriterInput?.markAsFinished()
audioAssetWriterInput?.markAsFinished()
}
assetWriter?.finishWriting() {
let status = assetWriter?.status
guard assetWriter?.error == nil else {
if let error = assetWriter?.error {
completion(nil, assetWriter?.error)
}
return
}
switch status {
case .completed?:
completion(assetWriter?.outputURL, nil)
case .failed?:
completion(nil, assetWriter?.error)
case .cancelled?:
completion(nil, assetWriter?.error)
default:
completion(nil, assetWriter?.error)
}
}
}
The problem
To restate the problem, this implementation creates four videos with virtually no issue in each video, however, the problem arises when I try to replay these videos in a sequential order using an AVQueuePlayer. One of these videos randomly freezes ONLY at the very start while continuing to play the audio until the end. I could perform this recording as many times as I like and receive different results. In fact, there are some instances where the freezing does NOT happen at all and plays all videos with no issues.
But more often than not, the freezing occurs at completely random intervals. This freezing does not unfreeze. The rest of the app is still responsive (I can press a button to delete all the videos and allow me to record all over again from the beginning). But performing the record again will cause a freeze at some other random time-interval.
I've tried merging the videos into a single-video and playing it in the Photos app, and the random freeze occurs there as well. BUT, I also download the videos separately to the Photos app as well (four short videos and single long video are downloaded to the Photos), and each video plays completely fine if played separately.
