How do I add transparency to a uint16 image in python (in the context of using connectedcomponents in opencv)?

Question

With a uint8, 3-channel image and uint8 binary mask, I have done the following in opencv and python in order to change an object on a black background into an object on a transparent background:

# Separate image into its 3 channels
b, g, r = cv2.split(img)
# Merge channels back with mask (resulting in a 4-channel image)
imgBGRA = cv2.merge((b, g, r, mask))

However, when I try doing this with a uint16, 3-channel image and uint16 binary mask, the saved result is 4-channel, but the background is still black. (I saved it as a .tiff file and viewed it in Photoshop.)

How can I make the background transparent, keeping the output image uint16?

UPDATE

Seeing @Shamshirsaz.Navid and @fmw42 comments, I tried imgBGRA=cv2.cvtColor(imgBGR, cv2.COLOR_BGR2BGRA). Then used Numpy to add the alpha channel from the mask: imgBGRA[:,:,3]=mask. (I hadn't tried this, as I thought that cvtColor operations required an 8-bit image.) Nonetheless, my results are the same.

I think the problem is my mask. When I run numpy.amin(mask), I get 0, and for numpy.amax(mask), I get 1. What should they be? I tried multiplying the mask by 255 prior to using the split/merge technique, but the background was still black. Then I tried mask*65535, but again the background was black.

I had tried to keep the scope of my initial post narrow. But it seems that my problem does lie somewhere in the larger scope of what I'm doing and how this uint16 mask gets created.

I'm using connectedComponentsWithStats (CC) to cut out the components on a uint16 image. CC requires an 8-bit mask, which I am using as input to CC. But the cutout results need to be from my uint16 original. This has required some alterations to the way I learned to use CC on uint8 images. Note that the per-component mask (which I eventually use to try to make the background transparent) is created as uint16. Here is the whittled down version:

# img is original image, dtype=uint16
# bin is binary mask, dtype=uint8
cc = cv2.connectedComponentsWithStats(bin, connectivity, cv2.CV_32S)
num_labels = cc[0]
labels = cc[1]
for i in range(1, num_labels):
    maskg = (labels == i).astype(np.uint16)   # with uint8: maskg = (labels == i).astype(np.uint8) * 255
    # NOTE: I don't understand why removing the `* 255` works; but after hours of experimenting, it's the only way I could get the original to appear correctly when saving 'glyph'; for all other methods I tried the colors were off in some significant way -- either grayish blue whereas the object in my original is variations of brown, or else a pixelated rainbow of colors)
    glyph = img * maskg[..., np.newaxis]      # with uint8: glyph = cv2.bitwise_and(img, img, mask=maskg)
    b, g, r = cv2.split(glyph)
    glyphBGRA = cv2.merge((b, g, r, maskg))

example (my real original image is huge and, also, I am not able share it; so I put together this example)

img (original uint16 image)

bin (input uint8 mask)

maskg (uint16 component mask created within loop) (this is a screenshot -- it shows up all black when uploaded directly)

glyph (img with maskg applied)

glyphBGRA (result of split and merge method trying to add transparency) (this is also a screenshot -- this one showed up all white/blank when added directly)

I hope this added info provides sufficient context for my problem.

Answer 1

I checked your last comment. I think an example might be better. Your code is correct; The question is, how did you use it? I attached a picture and a mask to test on them.

import sys,cv2

main = cv2.imread(sys.path[0]+'/main.png')
mask = cv2.imread(sys.path[0]+'/mask.png', cv2.IMREAD_GRAYSCALE)
mask = cv2.threshold(mask, 127, 255, cv2.THRESH_BINARY)[1]

b, g, r = cv2.split(main)
bgra = cv2.merge((b, g, r, mask))
cv2.imwrite(sys.path[0]+'/out_split_merge.png',bgra)

Main:

Mask:

Output:

If you open the final output with an image editing software, you will notice that part of it is transparent.

Answer 2

Diagnosis: Opencv is not able to save tiff with an alpha channel.

The following is from the opencv docs' entry for imwrite():

The function imwrite saves the image to the specified file. The image format is chosen based on the filename extension (see cv::imread for the list of extensions). In general, only 8-bit single-channel or 3-channel (with 'BGR' channel order) images can be saved using this function, with these exceptions:

• 16-bit unsigned (CV_16U) images can be saved in the case of PNG, JPEG 2000, and TIFF formats
• 32-bit float (CV_32F) images can be saved in PFM, TIFF, OpenEXR, and Radiance HDR formats; 3-channel (CV_32FC3) TIFF images will be saved using the LogLuv high dynamic range encoding (4 bytes per pixel)
• PNG images with an alpha channel can be saved using this function. To do this, create 8-bit (or 16-bit) 4-channel image BGRA, where the alpha channel goes last. Fully transparent pixels should have alpha set to 0, fully opaque pixels should have alpha set to 255/65535 (see the code sample below).

How I got to this point:

I manually removed the background in Photoshop and saved as png file and as tiff file. (They both look like this:)

Then I ran:

import cv2
import numpy as np
png16 = cv2.imread('c:/users/scott/desktop/python2/teststack/png16.png', cv2.IMREAD_UNCHANGED)
tif16 = cv2.imread('c:/users/scott/desktop/python2/teststack/tif16.tiff', cv2.IMREAD_UNCHANGED)
print('png16:', png16.dtype, png16.shape)
b, g, r, a = cv2.split(png16)
mmin = np.amin(a)
mmax = np.amax(a)
print('png16-a channel:', a.dtype, a.shape, mmin, mmax)
pixvals = np.unique(a.flatten()) # get all unique pixel values in a
print('png16-a channel pixel values:', pixvals)
print('tif16:', tif16.dtype, tif16.shape)
b, g, r, a = cv2.split(tif16)
mmin = np.amin(a)
mmax = np.amax(a)
print('tif16-a channel:', a.dtype, a.shape, mmin, mmax)
pixvals = np.unique(a.flatten()) # get all unique pixel values in a
print('tif16-a channel pixel values:', pixvals)
png16copy = png16.copy()
tif16copy = tif16.copy()
cv2.imwrite('c:/users/scott/desktop/python2/teststack/png16copy.png', png16copy)
cv2.imwrite('c:/users/scott/desktop/python2/teststack/tif16copy.tiff', tif16copy)

The output is all as one should expect:

png16: uint16 (312, 494, 4)
png16-a channel: uint16 (312, 494) 0 65535
png16-a channel pixel values: [    0 65535]
tif16: uint16 (312, 494, 4)
tif16-a channel: uint16 (312, 494) 0 65535
tif16-a channel pixel values: [    0 65535]

Back in Photoshop, the png file looked like it did before:

But the tiff file did not. Without alpha channel visible:

With alpha channel visible:

So I knew at this point that the problem was in the saving. I reread the opencv docs for imwrite and picked up on the logic: if it's not 8-bit single-channel or 3-channel, and if it's not spelled out explicitly in the exceptions, it won't work.

I did some more searching and found something that does work. I installed tifffile and ran:

from tifffile import imsave
tif16copy2 = cv2.cvtColor(tif16copy, cv2.COLOR_BGRA2RGBA)
imsave('c:/users/scott/desktop/python2/teststack/tif16copy2.tiff', tif16copy2)

Here is the result in Photoshop: