Separating the leaf from it's background

Question

I have a set of images, all of which look almost like this leaf here:

I want to extract the leaf from the background, for which I used the GrabCut algorithm as used here.

As a different approach, I also used thresholding based on ratios of r, g and b values as here:

import numpy as np
import cv2
import matplotlib.pyplot as plt

testImg = cv2.imread('path_to_the_image')
testImg = cv2.resize(testImg, (256, 256))
#bgImg = cv2.imread('')
#blurBg = cv2.GaussianBlur(bgImg, (5, 5), 0)
#blurBg = cv2.resize(blurBg, (256, 256))

#testImg = cv2.GaussianBlur(testImg, (5, 5), 0)
cv2.imshow('testImg', testImg)
#plt.imshow(bgImg)
cv2.waitKey(0)
#plt.show()

modiImg = testImg.copy()    
ht, wd = modiImg.shape[:2]

print(modiImg[0][0][0])

for i in range(ht):
    for j in range(wd):
        r = modiImg[i][j][0]
        g = modiImg[i][j][1]
        b = modiImg[i][j][2]

        r1 = r/g
        r2 = g/b
        r3 = r/b

        r4 = round((r1+r2+r3)/3, 1)

        if g > r and g > b:
            modiImg[i][j] = [255, 255, 255]
        elif r4 >= 1.2:
            modiImg[i][j] = [255, 255, 255]
        else:
            modiImg[i][j] = [0, 0, 0]

        # if r4 <= 1.1:
        #   modiImg[i][j] = [0, 0, 0]
        # elif g > r and g > b:
        #   modiImg[i][j] = [255, 255, 255]
        # else:
        #   modiImg[i][j] = [255, 255, 255]
        # elif r4 >= 1.2:
        #   modiImg[i][j] = [255, 255, 255]
        # else:
        #   modiImg[i][j] = [0, 0, 0]


plt.imshow(modiImg)
plt.show()

testImg = testImg.astype(float)

alpha = modiImg.astype(float) / 255

testImg = cv2.multiply(alpha, testImg)                

cv2.imshow('final', testImg/255)
cv2.waitKey(0)

But the dark spots on the leaf always go missing in the extracted leaf image as shown here:

Is there any other method to separate the leaf from its background, given that there is only one leaf per image, and the background is almost the same for other images that I have and also the leaves are positioned almost similarly as in here.

Anubhav Singh · Accepted Answer · 2019-07-28T16:21:55.040

You can try image segmentation using HSV colormap.

Code:

img =  cv2.imread('leaf.jpg')
hsv = cv2.cvtColor(img, cv2.COLOR_BGR2HSV)

# find the green color 
mask_green = cv2.inRange(hsv, (36,0,0), (86,255,255))
# find the brown color
mask_brown = cv2.inRange(hsv, (8, 60, 20), (30, 255, 200))
# find the yellow color in the leaf
mask_yellow = cv2.inRange(hsv, (21, 39, 64), (40, 255, 255))

# find any of the three colors(green or brown or yellow) in the image
mask = cv2.bitwise_or(mask_green, mask_brown)
mask = cv2.bitwise_or(mask, mask_yellow)

# Bitwise-AND mask and original image
res = cv2.bitwise_and(img,img, mask= mask)

cv2.imshow("original", img)
cv2.imshow("final image", res)
cv2.waitKey(0)
cv2.destroyAllWindows()

Output:

Moreover, If you change lower range of yellow color from (21, 39, 64) to (14, 39, 64), then you will see that the small black spots present on the leaf start filling and will improve the result even further.

Anno · Answer 2 · 2019-07-28T15:40:13.600

You may want to use a deep learning method. The U-Net is doing pretty good on tasks like that https://lmb.informatik.uni-freiburg.de/people/ronneber/u-net/. As I see they also provide a trained model. If you have Matlab and Caffee installed you should be able to copy your files into the right folder, run the program and receive the results you are looking for.

Thresholding is not a good idea for that kind of task. Your method should be able to recognize patterns instead of just looking at pixel's color.

A problem with the deep learning method is tough, that you either need a pretrained network that has trained the segmentation of RBG imges of leafes or you need data (RGB image of leafes and the corresponding segmentation).

Separating the leaf from it's background

2 Answers2