11

AVPlayer has a property called rate that is meant to control the playback rate. 1.0 is normal speed while values like 2.0 or 5.0 should playback at 2x and 5x respectively.

Whenever I set a playback rate value higher than 1.0 (say 10.0), the playback is very choppy and it looks like a large number of frames are getting dropped as the player can't keep up.

However, the same values in QuickTime Player (with the same movie), produce smooth playback for rates of 2x, 5x, 10x, 30x and 60x (as reported by the QuickTime Player).

I created a test OS X application that contains nothing more than an AVPlayerView and two buttons for setting the playback rate. A rate of 1.0 works as expected, but a rate of 10.0 produces very choppy playback.

However, the AVPlayerView has an odd quirk in that if you mouse-click on the playback timeline to seek to another location (while it's playing at 10x and choppy), then the AVPlayerView will "fix" the playback and the movie will be played smoothly at 10x. All it took was clicking on the playback timeline.

Does anyone know how to get smooth playback for rates other than 1x? It's obviously not a hardware problem or a file size problem because both QuickTime Player and AVPlayerView can do it.

Attempts

This question suggests that it might be an audio issue (and indeed both QuickTime Player and AVPlayerView mute the audio when forwarding) but all attempts on my part to either disable all audio tracks, mute all tracks or change the audio pitch algorithm did not seem to make a difference. Playback was still choppy even when there was no audio present.

I've also tried stopping playback and then calling prerollAtRate:completionHandler with the new rate but that doesn't make a difference either.

What is it that QuickTime Player and AVPlayerView are doing that allows for smooth movie playback at rates of 10x, 30x or even 60x?

Community
  • 1
  • 1
kennyc
  • 5,490
  • 5
  • 34
  • 57
  • 1
    Did you find any solution for this problem? If so , then what was that? – Abuzar Amin Aug 04 '16 at 09:17
  • Nope, sorry. I ended up having to build my own solution, but it's not as good as I would like. Long term, I think the best playback experience will require digging much deeper and handling the frame decoding outside of using AVPlayer. – kennyc Sep 09 '16 at 11:16
  • Under macos 10.12.3/Xcode 8.2.1, I have confirmed some of the above: speeds up to 2.0 exhibit no problem but setting `AVPlayer.rate` to a value greater than 2.0 causes choppy playback; using AVPlayerView to change the playback rate results in smooth playback for all speeds; setting the rate first to 0.0 and then calling `prerollAtRate(atRate:, completionHandler:)` and setting the rate high within the completion handler does not resolve the problem; creating an AVMutableComposition with only the video track does not resolve the problem; disabling the audio track does not resolve the problem. – Jamborino Feb 17 '17 at 10:49
  • It also might be worth noting that this is apparently a dark corner of AVFoundation even for Apple's documentation folks: The AVSimplePlayerOSX sample code provides a fast-forward button that exhibits this problem. – Jamborino Feb 17 '17 at 12:03
  • Possible duplicate of [Jerky playback from avplayer on Applying Rate greater than 2x](http://stackoverflow.com/questions/40506059/jerky-playback-from-avplayer-on-applying-rate-greater-than-2x) – kshitij godara Feb 17 '17 at 17:12
  • Not like it really matters, but that question was posted a year and a half after this question was posted. The solution proposed in that question might be effective under certain conditions, but not all. See my comment below... – kennyc Feb 18 '17 at 04:31
  • https://stackoverflow.com/a/33751843/5306470 –  Nov 09 '20 at 19:22

1 Answers1

2

This is only a workaround.

When the playback rate is changed from 0.0 to a large value, if this is the first zero-to-nonzero transition in playback rate since the last call to AVPlayer.replaceCurrentItem, playback is smooth (and audio is automatically muted). It is necessary that this is the first such transition: merely setting the rate to 0.0 first and then to the desired rate does not work.

So, for example, this will produce smooth playback at high speeds:

func setPlayerRate(player: AVPlayer, rate: Float) {
    // AVFoundation wants us to do most things on the main queue.
    DispatchQueue.main.async {
        if (rate == player.rate) {
            return
        }
        if (rate > 2.0 || rate < -2.0) {
            let playerItem = player.currentItem
            player.replaceCurrentItem(with: nil)
            player.replaceCurrentItem(with: playerItem)
            player.rate = rate
        } else {
            // No problems "out of the box" with rates in the range [-2.0,2.0].
            player.rate = rate
        }
    }
}
Jamborino
  • 161
  • 9
  • Swapping out the AVPlayerItem, or re-timing media tracks in an AVComposition are both valid, if not heavy-handed, ways at getting smoother playback. However, they aren't great solutions if you want to allow the user to change the playback rate during live playback. Such behaviour is common in video applications that have a "jog" dial. Introspection of QuickTime Player suggests (though isn't confirmed) that for some playback rates it is making repeated calls to `seekTime...`, likely slaved off a custom timeBase and maybe doing some decoding/caching trickery for better performance. – kennyc Feb 18 '17 at 04:39
  • ...And changing playback speed with ease is indeed something that I'd like to be able to do! Now that you mention it, I have _also_ built a thing that shows movie previews the same way that Final Cut Pro X does, where as you mouse over them, cells in a sheet of thumbnails show the video. I've felt frustrated that it worked so well, even when flinging the mouse around, and this did not. I was just not smart enough to think of applying seekTime in this way. Thanks! I will give this a try. That will also circumvent the bug that some notifications go missing when playing back at high speed. – Jamborino Feb 18 '17 at 08:34
  • For a small video thumbnail that the user can scrub through by just moving the mouse across it, you'll probably get more than adequate performance by playing around with the time you seek to and the allowable tolerance. Decoding is almost instantaneous if you seek to a "key frame" rather than an arbitrary time. Since timing precision isn't really important in a video preview, you might want to consider massaging the time you seek to so that it falls on, or very close to, a key frame. See TechNote `TN2404` for more details on using `AVSampleCursor` to identify the keyframes. – kennyc Feb 20 '17 at 14:04