How to replace a checked pattern in a PNG image with transparent in Python?

Question

I am trying to remove the checkered background (which represents transparent background in Adobe Illustrator and Photoshop) with transparent color (alpha channel) in some PNGs with Python script.

First, I use template matching:

import cv2
import numpy as np
from matplotlib import pyplot as plt

img_rgb = cv2.imread('testimages/fake1.png', cv2.IMREAD_UNCHANGED)
img_gray = cv2.cvtColor(img_rgb, cv2.COLOR_BGR2GRAY)
template = cv2.imread('pattern.png', 0)

w, h = template.shape[::-1]
res = cv2.matchTemplate(img_gray, template, cv2.TM_CCOEFF_NORMED)
threshold = 0.8
loc = np.where( res >= threshold)

for pt in zip(*loc[::-1]):
    if len(img_rgb[0][0]) == 3:
        # add alpha channel
        rgba = cv2.cvtColor(img_rgb, cv2.COLOR_RGB2RGBA)
        rgba[:, :, 3] = 255 # default not transparent
        img_rgb = rgba
    # replace the area with a transparent rectangle
    cv2.rectangle(img_rgb, pt, (pt[0] + w, pt[1] + h), (255, 255, 255, 0), -1) 

cv2.imwrite('result.png', img_rgb)

Source Image: fake1.png

Pattern Template: pattern.png

Output: result.png (the gray area is actually transparent; enlarge a bit for viewing easier)

I know this approach has problems, as the in some cases, the template cannot be identified fully, as part of the pattern is hidden by the graphics in the PNG image.

My question is: How can I match such a pattern perfectly using OpenCV? via FFT Filtering?

References:

• How particular pixel to transparent in opencv python?
• Detecting a pattern in an image and retrieving its position
• https://python.plainenglish.io/how-to-remove-image-background-using-python-6f7ffa8eab15
• https://answers.opencv.org/question/232506/make-the-background-of-the-image-transparent-using-a-mask/
• https://dsp.stackexchange.com/questions/36679/which-image-filter-can-be-applied-to-remove-gridded-pattern-from-corrupt-jpegs

Answer 1

Here is one way to do that in Python/OpenCV simply by thresholding on the checks color range.

Input:

import cv2
import numpy as np

# read input
img = cv2.imread("fake.png")

# threshold on checks
low = (230,230,230)
high = (255,255,255)
mask = cv2.inRange(img, low, high)

# invert alpha
alpha = 255 - mask

# convert img to BGRA
result = cv2.cvtColor(img, cv2.COLOR_BGR2BGRA)
result[:,:,3] = alpha

# save output
cv2.imwrite('fake_transparent.png', result)

cv2.imshow('img', img)
cv2.imshow('mask', mask)
cv2.imshow('result', result)
cv2.waitKey(0)
cv2.destroyAllWindows()

Download the resulting image to see that it is actually transparent.

Answer 2

since you're working on PNG's with transparent backgrounds, it would probably be equally viable to instead of trying to detect the checkered background, you try to extract the stuff that isn't checkered. This could probably be achieved using a color check on all pixels. You could use opencv's inRange() function. I'll link a StackOverflow link below that tries to detect dark spots on a image. Inrange example

Answer 3

Here is one way to use DFT to process the image in Python/OpenCV/Numpy. One does need to know the size of the checkerboard pattern (light or dark square size).

• Read the input
• Separate channels
• Apply DFT to each channel
• Shift origin from top left to center of each channel
• Extract magnitude and phase images from each channel
• Define the checkerboard pattern size
• Create a black and white checkerboard image of the same size
• Apply similar DFT processing to the checkerboard image
• Get the spectrum from the log(magnitude)
• Threshold the spectrum to form a mask
• Zero out the DC center point in the mask
• OPTION: If needed apply morphology dilate to thicken the white dots. But does not seem to be needed here
• Invert the mask so the background is white and the dots are black
• Convert the mask to range 0 to 1 and make 2 channels
• Apply the two-channel mask to the center shifted DFT channels
• Shift the center back to the top left in each masked image
• Do the IDFT to get back from complex domain to real domain on each channel
• Merge the resulting channels back to a BGR image as the final reconstituted image
• Save results

Input:

import numpy as np
import cv2
import math

# read input 
# note: opencv fft only works on grayscale
img = cv2.imread('fake.png')
hh, ww = img.shape[:2]

# separate channels
b,g,r = cv2.split(img)

# convert images to floats and do dft saving as complex output
dft_b = cv2.dft(np.float32(b), flags = cv2.DFT_COMPLEX_OUTPUT)
dft_g = cv2.dft(np.float32(g), flags = cv2.DFT_COMPLEX_OUTPUT)
dft_r = cv2.dft(np.float32(r), flags = cv2.DFT_COMPLEX_OUTPUT)

# apply shift of origin from upper left corner to center of image
dft_b_shift = np.fft.fftshift(dft_b)
dft_g_shift = np.fft.fftshift(dft_g)
dft_r_shift = np.fft.fftshift(dft_r)

# extract magnitude and phase images
mag_b, phase_b = cv2.cartToPolar(dft_b_shift[:,:,0], dft_b_shift[:,:,1])
mag_g, phase_g = cv2.cartToPolar(dft_g_shift[:,:,0], dft_g_shift[:,:,1])
mag_r, phase_r = cv2.cartToPolar(dft_r_shift[:,:,0], dft_r_shift[:,:,1])

# set check size (size of either dark or light square)
check_size = 15

# create checkerboard pattern
white = np.full((check_size,check_size), 255, dtype=np.uint8)
black = np.full((check_size,check_size), 0, dtype=np.uint8)
checks1 = np.hstack([white,black])
checks2 = np.hstack([black,white])
checks3 = np.vstack([checks1,checks2])
numht = math.ceil(hh / (2*check_size))
numwd = math.ceil(ww / (2*check_size))
checks = np.tile(checks3, (numht,numwd))
checks = checks[0:hh, 0:ww]

# apply dft to checkerboard pattern
dft_c = cv2.dft(np.float32(checks), flags = cv2.DFT_COMPLEX_OUTPUT)
dft_c_shift = np.fft.fftshift(dft_c)
mag_c, phase_c = cv2.cartToPolar(dft_c_shift[:,:,0], dft_c_shift[:,:,1])

# get spectrum from magnitude (add tiny amount to avoid divide by zero error)
spec = np.log(mag_c + 0.00000001)

# theshold spectrum
mask = cv2.threshold(spec, 1, 255, cv2.THRESH_BINARY)[1]

# mask DC point (center spot)
centx = int(ww/2)
centy = int(hh/2)
dot = np.zeros((3,3), dtype=np.uint8)
mask[centy-1:centy+2, centx-1:centx+2] = dot

# If needed do morphology dilate by small amount. 
# But does not seem to be needed in this case

# invert mask
mask = 255 - mask

# apply mask to real and imaginary components
mask1 = (mask/255).astype(np.float32)
mask2 = cv2.merge([mask1,mask1])
complex_b = dft_b_shift*mask2
complex_g = dft_g_shift*mask2
complex_r = dft_r_shift*mask2

# shift origin from center to upper left corner
complex_ishift_b = np.fft.ifftshift(complex_b)
complex_ishift_g = np.fft.ifftshift(complex_g)
complex_ishift_r = np.fft.ifftshift(complex_r)

# do idft with normalization saving as real output and crop to original size
img_notch_b = cv2.idft(complex_ishift_b, flags=cv2.DFT_SCALE+cv2.DFT_REAL_OUTPUT)
img_notch_b = img_notch_b.clip(0,255).astype(np.uint8)
img_notch_b = img_notch_b[0:hh, 0:ww]
img_notch_g = cv2.idft(complex_ishift_g, flags=cv2.DFT_SCALE+cv2.DFT_REAL_OUTPUT)
img_notch_g = img_notch_g.clip(0,255).astype(np.uint8)
img_notch_g = img_notch_g[0:hh, 0:ww]
img_notch_r = cv2.idft(complex_ishift_r, flags=cv2.DFT_SCALE+cv2.DFT_REAL_OUTPUT)
img_notch_r = img_notch_r.clip(0,255).astype(np.uint8)
img_notch_r = img_notch_r[0:hh, 0:ww]

# combine b,g,r components
img_notch = cv2.merge([img_notch_b, img_notch_g, img_notch_r])

# write result to disk
cv2.imwrite("fake_checks.png", checks)
cv2.imwrite("fake_spectrum.png", (255*spec).clip(0,255).astype(np.uint8))
cv2.imwrite("fake_mask.png", mask)
cv2.imwrite("fake_notched.png", img_notch)

# show results
cv2.imshow("ORIGINAL", img)
cv2.imshow("CHECKS", checks)
cv2.imshow("SPECTRUM", spec)
cv2.imshow("MASK", mask)
cv2.imshow("NOTCH", img_notch)
cv2.waitKey(0)
cv2.destroyAllWindows()

Checkerboard image:

Spectrum of checkerboard:

Mask:

Result (notch filtered image):

The checkerboard pattern in the result is mitigated from the original, but still there upon close inspection.

From here one needs to threshold on the white background and invert to make an image for the alpha channel. Then convert the image to 4 BGRA and insert the alpha channel into the BGRA image as I described in my other answer below.