1

I am running into weird performance issues on older device (iPad Mini 2 of year 2014) ever since migrating to MTKView from OpenGLES based views. To summarize, I am rendering live video preview using MTKView and also optionally record videos (like in RosyWriter sample code). My new app version is written in Swift and supports both Metal & OpenGLES. So I have both Open GL and Metal rendering code to compare. I also have a legacy version of app written in Objective C & uses OpenGLES for all preview and rendering. Here are performance issues I see on iPad Mini 2:

  1. When using MTKView, if I debug the app in XCode and zoom video (using AVCaptureDevice zoom API, no Metal code to crop/zoom frames), the frame rate drops significantly (from 30 to 10 fps). If I record video while zooming (which means additional video encoding pipeline in another thread), there are frame drops in the recorded video as well.

  2. When I profile the app using time profiler, MTKView is smooth enough with zoom. However, if I start recording video now , MTKView fps again drops or frames get delayed in the preview, but there is no frame drop issue in recorded video -- the end recorded video is still smooth.

  3. If I select Release version of app for Debug, behavior is same as while profiling,

  4. If I switch back to GLKView instead of MTKView, there are no issues in preview even in Debug version. There are still issues when recording -- preview gets delayed when zooming. But if I profile the app, there are NO delays or frame drops!!!!

  5. Finally, the original legacy version in Objective C that uses OpenGLES for all purposes has no issues at all.

Now the question is what tools in Instruments can I use to nail down the exact issues for frame drops in preview and recording. Can Metal not match performance of OpenGLES on older devices?

Deepak Sharma
  • 5,577
  • 7
  • 55
  • 131
  • As with all performance-related issues you need to profile the app to see where the bottle necks are. Fire-up `Instruments` and see where your issues are. – trojanfoe Oct 13 '19 at 09:09
  • The whole issue is when running Instruments with GLKView there are no issues (see point 4). – Deepak Sharma Oct 13 '19 at 09:24
  • Seems to be problem with Metal rendering and previewing paths. How do I diagnose the issues? – Deepak Sharma Oct 13 '19 at 09:26
  • I am not quite sure what you mean by "previewing paths", however give the [Instruments Metal System Trace](https://developer.apple.com/documentation/metal/using_metal_system_trace_in_instruments_to_profile_your_app?language=objc) a go. – trojanfoe Oct 13 '19 at 09:28
  • 1
    Please provide the source code that describes your problem. – Hamid Yusifli Oct 13 '19 at 11:49
  • Ok, will provide. I believe it's standard code to display CMSampleBuffer using Metal. – Deepak Sharma Oct 13 '19 at 12:57
  • Could you please provide the iOS version of the iPAD Mini 2? In my experience, pre iOS 11 Metal API was somewhat buggy overall. – Vlad Alex Jan 17 '20 at 11:30

0 Answers0