Image foreground extraction techniques using Python

Question

I've prepared a piece of code in python using OpenCV that extracts the foreground in the image. My program is getting a stream of images and I need to extract the object from those images.

The images include products like shoes, bags or clothing. Most of the images have white background but sometimes I get a grey gradient background as well. I've used thresholding to separate the foreground from the background and extract the object.

But the problem is that even if I assume that the background will be white and set a fixed upper and lower thresholding values, I sometimes get whites in the object and thresholding doesn't render the correct output.

I wanted to know what might be some good approaches in Computer Vision that I can utilize to get the best results here?

You will need AI / Deep Learning to do that reasonably well. See for example, http://remove.bg — fmw42, Apr 27 '21 at 00:42
@fmw42 I was thinking of the same solution. Does there happen to be some datasets available for this test case? — Ubaid, Apr 27 '21 at 21:05
Just did a search and found https://towardsdatascience.com/background-removal-with-deep-learning-c4f2104b3157. It uses the COCO data set. — fmw42, Apr 27 '21 at 21:19

score 1 · Answer 1 · answered Apr 27 '21 at 12:25

You can get this result:

using the canny edge detector, dilation and erosion:

import cv2
import numpy as np

def process(img):
    img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    img_canny = cv2.Canny(img_gray, 12, 54)
    kernel = np.ones((3, 3))
    img_dilate = cv2.dilate(img_canny, kernel, iterations=10)
    img_erode = cv2.erode(img_dilate, kernel, iterations=8)
    return img_erode

img = cv2.imread("robe.jpg")
contours, _ = cv2.findContours(process(img), cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_NONE)
cv2.drawContours(img, contours, -1, (0, 255, 0), 2)
cv2.imshow("Image", img)
cv2.waitKey(0)

Explanation of the process function:

First, convert the image to grayscale in order to use the canny edge detector on it. Then, detect its edges using the canny edge detector:

def process(img):
    img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
    img_canny = cv2.Canny(img_gray, 12, 54)

Define a kernel to use to dilate and erode the image:

    kernel = np.ones((3, 3))

Finally, dilate the image several iterations to fill in gaps, remove noise and smooth out bumps in the edges. of course, dilation makes edges thicker, so after that, erode the image to shrink them back down. But as there was a lot of gaps in the edges before dilation, erode the image by less iterations. Return the eroded image:

    img_dilate = cv2.dilate(img_canny, kernel, iterations=10)
    img_erode = cv2.erode(img_dilate, kernel, iterations=8)
    return img_erode

I've tried it and a couple of other edge detectors as well but the problem with edge detectors is that we only get the outer contour and this is a fairly simple image in which there are no inner contours that need to be detected. Let's say if there was a handbag in this image and there was some part of background inside the bags strap we wouldn't be getting that using edge detectors. And if we include inner contours as well, we'd certainly be getting a lot of unwanted contours and while doing so we might lose some part of the object as well. — Ubaid, Apr 27 '21 at 21:15

Image foreground extraction techniques using Python

1 Answers1