Convert CGImageRef to PIL

Question

How could I convert a CGImageRef to PIL without saving the image to disk on osx?

I though about getting the raw pixel data from the CGImageRef and using Image.fromstring() to make the PIL image by doing

import mss
import Quartz.CoreGraphics as CG
from PIL import Image

mss = mss.MSSMac()
for i, monitor in enumerate(mss.enum_display_monitors(0)):
    imageRef = mss.get_pixels(monitor)
    pixeldata = CG.CGDataProviderCopyData(CG.CGImageGetDataProvider(imageRef))
    img = Image.fromstring("RGB", (monitor[b'width'], monitor[b'height']), pixeldata)
    img.show()

but this doesn't give me the correct image.

This is the image I expect:

enter image description here

and this is the image I get in PIL:

enter image description here

Could you please add a minimal yet complete and reproducible scenario for us to test? Perhaps you will get better answers if you do. — Rachcha, Feb 28 '15 at 06:48
What does *does not give a correct image* mean, is it just random junk or does it look a bit like your image? If the latter you could at least submit the expected and actual image. — Antti Haapala -- Слава Україні, Feb 28 '15 at 06:49

Thomas Marrinan · Answer 1 · 2015-05-22T14:47:05.830

The screencapture from CG doesn't necessarily use the RGB colorspace. It may use RGBA or something else. Try changing:

img = Image.fromstring("RGB", (monitor[b'width'], monitor[b'height']), pixeldata)

to

img = Image.fromstring("RGBA", (monitor[b'width'], monitor[b'height']), pixeldata)

Here is how I detect which colorspace is actually being captured:

bpp = CG.CGImageGetBitsPerPixel(imageRef)
info = CG.CGImageGetBitmapInfo(imageRef)
pixeldata = CG.CGDataProviderCopyData(CG.CGImageGetDataProvider(imageRef))

img = None
if bpp == 32:
    alphaInfo = info & CG.kCGBitmapAlphaInfoMask
    if alphaInfo == CG.kCGImageAlphaPremultipliedFirst or alphaInfo == CG.kCGImageAlphaFirst or alphaInfo == CG.kCGImageAlphaNoneSkipFirst:
        img = Image.fromstring("RGBA", (CG.CGImageGetWidth(imageRef), CG.CGImageGetHeight(imageRef)), pixeldata, "raw", "BGRA")
    else:
        img = Image.fromstring("RGBA", (CG.CGImageGetWidth(imageRef), CG.CGImageGetHeight(imageRef)), pixeldata)
elif bpp == 24:
    img = Image.fromstring("RGB", (CG.CGImageGetWidth(imageRef), CG.CGImageGetHeight(imageRef)), pixeldata)

score 0 · Answer 2 · answered May 09 '17 at 09:25

It was a bug I fixed some time ago. Here is how to achieve what you want using the latest mss version (2.0.22):

from mss.darwin import MSS
from PIL import Image

with MSS() as mss:
    for monitor in mss.enum_display_monitors(0):
        pixeldata = mss.get_pixels(monitor)
        img = Image.frombytes('RGB', (mss.width, mss.height), pixeldata)
        img.show()

Note that pixeldata is just a reference to mss.image, you can use it directly.

Convert CGImageRef to PIL

2 Answers2