2

What is the main difference between MTLTexture vs CGImageRef? When do we need to use MTLTexture instead of CGImageRef (and vice versa)?

I have an app (say a video game) that draw everything by itself on a dedicated surface. this includes animation at 60fps (so I need to redraw the surface every 16ms). I don't know the most efficient way to do my app using Metal

zeus
  • 12,173
  • 9
  • 63
  • 184

1 Answers1

3

First of all, MTLTexture comes from a low-level graphics API. MTLTexture refers to an "image" that resides in memory accessible to GPU (no necessarily on GPU itself). You can then write a program that uses Metal, specifically render (MTLRenderPipelineState) or compute (MTLComputePipelineState) pipeline states that contain shader (programs that run on GPU) to read textures, sample them, write to them and use them as attachments (output rendering results to them). Textures can also be copied to buffers (MTLBuffer) and other textures, if you want to read back texture data on the CPU. But MTLTexture is mostly intended to be used by GPU rather than CPU. Also, MTLTexture is not limited to being 2D, it can also be a cube texture or even a 3D texture.

CGImage, on the other hand, comes from a higher-level API (Core Graphics or Quartz 2D) that is intended for 2D use. You don't need shaders or GPU pipeines to create or modify CGImages and there are many functions to work with these images "out of the box".

I would say, if you have a 3D video game, you can check out Metal, but it's a low level API, and setting up Metal is a much more involved process than setting up OpenGL, for example. You can't use Core Graphics for 3D games as-is. If Metal seems too hard, you can check out higher-level APIs from Apple, such as SceneKit, which are also intended for game development.

I can't say much about 2D game development, but you can definitely use Metal for it, it might just be a bit "overkill".

In conclusion, you need to find a balance between complexity and control and chose what best suits you.

JustSomeGuy
  • 3,677
  • 1
  • 23
  • 31
  • Thanks Egor! But in fact if I use CGImage, I will also use metal no ? – zeus Apr 23 '19 at 17:25
  • Don't assume a `CGImage` will use Metal. It may or may not. That's an implementation detail. But a `MTLTexture` always will. – user1118321 Apr 24 '19 at 03:04
  • Yes, it's a higher level API and it abstracts implementation details, so as @user1118321 above said, it can use `MTLTexture` or something else underneath, it's implementation detail and not really relevant if you use Core Graphics. – JustSomeGuy Apr 24 '19 at 08:31
  • If you want the best possible performance then use Metal directly. The CoreGraphics APIs are older and typically execute on the CPU as opposed to the GPU. The bottleneck of rendering at 60 FPS is going to be the H.264 decoding logic that generates pixel buffers from the H.264 encoded source stored in a file. What you can expect on A8 and newer metal hardware is about 2ms render times for full screen videos. – MoDJ May 28 '19 at 17:12