How to avoid Airplay to freeze when using AVComposition?

Question

My code is as follows:

//init mix composition
let mixComposition = AVMutableComposition()

//video asset
let videoAsset = AVURLAsset(url: VIDEO_URL)
let videoTrack = mixComposition.addMutableTrack(withMediaType: AVMediaTypeVideo, preferredTrackID: kCMPersistentTrackID_Invalid)
try? videoTrack.insertTimeRange(CMTimeRangeMake(kCMTimeZero, videoAsset.duration), of: videoAsset.tracks(withMediaType: AVMediaTypeVideo).first!, at: kCMTimeZero)

//audio track
let audioTrack = mixComposition.addMutableTrack(withMediaType: AVMediaTypeAudio, preferredTrackID: kCMPersistentTrackID_Invalid)
try? audioTrack.insertTimeRange(CMTimeRangeMake(kCMTimeZero, videoAsset.duration), of: videoAsset.tracks(withMediaType: AVMediaTypeAudio).first!, at: kCMTimeZero)

//subtitles
let path = Bundle.main.path(forResource: "subtitle", ofType: "vtt")!
let subtitleAsset = AVAsset(url: URL(fileURLWithPath: path))
let subtitleTrack = mixComposition.addMutableTrack(withMediaType: AVMediaTypeText, preferredTrackID: kCMPersistentTrackID_Invalid)
try? subtitleTrack.insertTimeRange(CMTimeRangeMake(kCMTimeZero, videoAsset.duration), of: subtitleAsset.tracks(withMediaType: AVMediaTypeText)[0], at: kCMTimeZero)

 //setup player and play
 let item = AVPlayerItem(asset: videoAsset)
 let player = AVPlayer(playerItem: item)
 self.player = player
 self.player?.allowsExternalPlayback = true
 self.player?.usesExternalPlaybackWhileExternalScreenIsActive = true
 self.player?.play()

When asked if the videoAsset.isCompatibleWithAirPlayVideo it returns true. When you load the application it all works smoothly until you enter Airplay mode, the screen freezes until you go out of Airplay mode again.

This is completely different behaviour if you were to change mixComposition to videoAsset. Now it all works, but you won't have the external subtitles.

Did anyone else run into this problem and what the work-around. My end goal is to include external subtitles to mp4 video.

Answer 1

I have code in my App almost identical to yours and it works in Airplay-mode so I think you are almost there.

I load the video, audio and vtt-subtitles from webserver and had two major challenges:

1. Audio could not play unless it came from https-host regardless of plist settings (video played fine from http-host)
2. Subtitles only lasted 10-12 minutes of a movie. Solution was to download subtitle and load from .documentDirectory using FileManager

The only difference from your code is that I initialize an AVPlayerViewController and set the player to the AVPlayer:

var moviePlayer = AVPlayerViewController()
moviePlayer.player = player

and then present the moviePlayer

self.view.window?.rootViewController?.present(self.moviePlayer, animated: true, completion : { self.moviePlayerFinishedLoading() } 
self.moviePlayer.player?.play()

Hope this helps