UIImage created from MTKView results in color/opacity differences

Question

When I capture the contents of an MTKView into a UIImage, the resulting image looks qualitatively different, as shown below:

The code I use to generate the UIImage is as follows:

let kciOptions = [kCIContextWorkingColorSpace: CGColorSpace(name: CGColorSpace.sRGB)!,
                         kCIContextOutputPremultiplied: true,
                         kCIContextUseSoftwareRenderer: false] as [String : Any]
let lastDrawableDisplayed = self.currentDrawable! // needed to hold the last drawable presented to screen
drawingUIView.image = UIImage(ciImage: CIImage(mtlTexture: lastDrawableDisplayed.texture, options: kciOptions)!)

Since I don't modify the ciImage orientation (.oriented(CGImagePropertyOrientation.downMirrored)) the resulting image is upside down, as shown in the image above. I leave the mirrored orientation as is so I can point out the color differences between the two image captures.

No matter how I change the kciOptions parameters, (say, even changing the colorspace to grayscale) I'm not seeing any changes in the resulting UIImage, which appears much more dim/desaturated than the original. Does anybody have any suggestions for how I can accurately capture what I'm drawing on MTKView to an UIImage? Any suggestions would be much appreciated.

Below are my MTKView settings which may prove relevant:

let renderPipelineDescriptor = MTLRenderPipelineDescriptor()
renderPipelineDescriptor.vertexFunction = vertexProgram
renderPipelineDescriptor.sampleCount = self.sampleCount
renderPipelineDescriptor.colorAttachments[0].pixelFormat = MTLPixelFormat.bgra8Unorm
renderPipelineDescriptor.colorAttachments[0].isBlendingEnabled = true
renderPipelineDescriptor.colorAttachments[0].rgbBlendOperation = .add
renderPipelineDescriptor.colorAttachments[0].alphaBlendOperation = .add
renderPipelineDescriptor.colorAttachments[0].sourceRGBBlendFactor = .sourceAlpha         renderPipelineDescriptor.colorAttachments[0].destinationRGBBlendFactor = .oneMinusSourceAlpha            renderPipelineDescriptor.colorAttachments[0].sourceAlphaBlendFactor = .sourceAlpha              renderPipelineDescriptor.colorAttachments[0].destinationAlphaBlendFactor = .oneMinusSourceAlpha
self.isOpaque = false // makes MTKView bg transparent

score 4 · Answer 1 · answered Jul 13 '18 at 04:26

I've seen this issue in a couple of posts, but no clear answer. Here is what I've found:

For starters,

renderPipelineDescriptor.colorAttachments[0].pixelFormat = MTLPixelFormat.bgra8Unorm

should really just be set to the MTKView's native pixel format

renderPipelineDescriptor.colorAttachments[0].pixelFormat = self.colorPixelFormat

Secondly, when I set the CIImage's options:

let kciOptions = [kCIContextWorkingColorSpace: CGColorSpace(name: CGColorSpace.sRGB)!,
                         kCIContextOutputPremultiplied: true,
                         kCIContextUseSoftwareRenderer: false] as [String : Any]

It didn't matter what I set kCIContextWorkingColorSpace to, I never saw any visual difference regardless of what I used. The property I really needed to set is called KCIImageColorSpace. So the updated kciOptions looks like:

let kciOptions = [kCIImageColorSpace: CGColorSpaceCreateDeviceRGB(),
                      kCIContextOutputPremultiplied: true,
                      kCIContextUseSoftwareRenderer: false] as [String : Any]

In a similar way of using the view's native pixel format, calling CGColorSpaceCreateDeviceRGB() creates an RGB colorspace that is specific to the device being used.

Adding color space option did not work for me. Also additg CIContextOption there is not allowed now. — user5865651, Jun 27 '19 at 12:41
Converting texture's format before getting CIImage worked for me. Like: let tex = drawable!.texture.makeTextureView(pixelFormat: .bgra8Unorm_srgb)! — user5865651, Jun 27 '19 at 13:15
Thank you, I'm also building a drawing engine and all the questions you made have helped a lot — Ebarella, May 31 '20 at 16:40

score 3 · Answer 2 · answered Jun 13 '18 at 11:18

3

let context = CIContext()
let texture = metalView.currentDrawable!.texture
let cImg = CIImage(mtlTexture: texture, options: nil)!
let cgImg = context.createCGImage(cImg, from: cImg.extent)!
let uiImg = UIImage(cgImage: cgImg)

answered Jun 13 '18 at 11:18

Ruban4Axis

821
6
27

4

Your answer would be better if you explained why this code works, but thanks for contributing! – rst-2cv Jun 13 '18 at 11:35

score 2 · Accepted Answer · answered Jun 13 '18 at 23:00

2

Your CGColorSpace is .sRGB but your renderPipelineDescriptor's pixelFormat is .bgra8Unorm. Try changing that line to:

renderPipelineDescriptor.colorAttachments[0].pixelFormat = MTLPixelFormat.bgra8Unorm_srgb

answered Jun 13 '18 at 23:00

hartw

356
3
17

1

Thank you @hartw. Your answer was very nearly the reason for the color difference. After much heartache, I came to a subtle finding which I'll post below. Thank you again – Plutovman Jul 13 '18 at 04:07

UIImage created from MTKView results in color/opacity differences

3 Answers3

Linked