how can I detect a certain shape (plant stem) in a binary image?

Question

I'm working on a shape based leaf classification project when i got stuck for hours trying to figure out how to rotate the image based on the stem of the leaf.

Here is an example of an input image.

sample

I have been trying to apply morphological transformations with opencv and filter2D with different kernels and even HoughLines, this is the closes I have gotten so far.

detected-steam

Any help would be appreciated, thanks in advance.

EDIT

The position of the stem is important because of the different kind of leaves I'm trying to classify, so What I'm trying to acomplish is to get the leaf in a vertical position with the stem being at the bottom.

The images I provided are the thresholded images, I leave the original one down here.

I provide a second sample because I in this particular case i couldn't get the stem at the bottom so it ended up being at the top.

Sample returning 2 degrees.

If you KNOW the leaf is always in one of the cardinal directions, then you can sum up the number of "inside" pixels in each column, an each row, and from that list it should be clear which direction has your "long thin" section. — Tim Roberts, Apr 19 '21 at 20:14
Post your original image without the tick marks and labels so we can work with it. You might consider using the equivalent ellipse orientation from image moments to get the direction. — fmw42, Apr 19 '21 at 20:24

fmw42 · Answer 1 · 2021-04-20T18:28:49.800

1

Here is one idea. Fit an ellipse to thresholded leaf shape (assuming it will be longer than wide).

Input:

import cv2
import numpy as np

# read image
img = cv2.imread('leaf.png')

# threshold on color
lower=(84,1,68)
upper=(84,1,68)
thresh = cv2.inRange(img, lower, upper)

# get contour
contours = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
big_contour = max(contours, key=cv2.contourArea)

# fit ellipse to leaf contours
ellipse = cv2.fitEllipse(big_contour)
(xc,yc), (d1,d2), angle = ellipse
print(xc,yc,d1,d1,angle)

# draw ellipse on copy of input
result = img.copy() 
cv2.ellipse(result, ellipse, (0,0,255), 1)

# save results
cv2.imwrite('leaf_threshold.png',thresh)
cv2.imwrite('leaf_ellipse.png',result)

# show results
cv2.imshow("thresh", thresh)
cv2.imshow("result", result)
cv2.waitKey(0)
cv2.destroyAllWindows()

Threshold image:

Ellipse image:

\

Ellipse Information:

126.44944763183594 101.98369598388672 112.40930938720703 112.40930938720703 89.33087158203125

So angle = 89.33087158203125 deg (cw from -y axis, i.e. from the top) or 
   angle = 0.66912841796875 deg (ccw from the x axis, i.e. from right side)

ADDITION:

Here is a more complete solution. But it assumes the leaf will be longer than wide so that the ellipse major axis aligns along the step direction.

Leaf 1:

Leaf 2:

import cv2
import numpy as np

# read image
#img = cv2.imread('leaf1.jpg')
img = cv2.imread('leaf2.jpg')

# threshold on color
lower=(0,0,0)
upper=(130,190,140)
thresh = cv2.inRange(img, lower, upper)

# get contour
contours = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
contours = contours[0] if len(contours) == 2 else contours[1]
big_contour = max(contours, key=cv2.contourArea)

# fit ellipse to leaf contours
ellipse = cv2.fitEllipse(big_contour)
(xc,yc), (d1,d2), angle = ellipse
print(xc,yc,d1,d1,angle)

# draw ellipse on copy of input
graphic = img.copy() 
cv2.ellipse(graphic, ellipse, (0,0,255), 1)


# rotate image so step points downward   
if angle >= 135 and angle <=180:
    result = cv2.rotate(img, cv2.ROTATE_180)
elif angle >= 45 and angle <135:
    result = cv2.rotate(img, cv2.ROTATE_90_CLOCKWISE)
elif angle >= 0 and angle <45:
    result = img.copy()
    

# save results
cv2.imwrite('leaf2_threshold.png',thresh)
cv2.imwrite('leaf2_ellipse.png',graphic)
cv2.imwrite('leaf2_result.png',result)

# show results
cv2.imshow("thresh", thresh)
cv2.imshow("graphic", graphic)
cv2.imshow("result", result)
cv2.waitKey(0)
cv2.destroyAllWindows()

Leaf 1 Threshold:

Leaf 1 Ellipse:

Leaf 1 Rotated:

Leaf 2 Threshold:

Leaf 2 Ellipse:

Leaf 2 Rotated:

edited Apr 20 '21 at 18:28

answered Apr 19 '21 at 20:47

fmw42

46,825
10
62
80

Hi fmw42, thank you for you very quick answer, you code works but there is something i left out, I need the stem to be vertical and at placed at the bottom of the image. How can I determine wether to convert the angle to negative or not? Thanks in advance – M. Villanueva Apr 20 '21 at 07:33
Rotate this image by angle = 89.33087158203125 deg or angle = -89.33087158203125 deg. Whichever one gives you the orientation you want, keep that sign for all the other ones. – fmw42 Apr 20 '21 at 15:42
I don't understand what is the criteria for negative or positive angle, Using the same angle for the two images in the description with positive angle the stem ends up at the top of the image. Some other images I tried dont even rotate (I'm getting 2 degrees when I expect 90 or -90) – M. Villanueva Apr 20 '21 at 17:14
Post an example that gives 2 degrees. Is the leaf wider that long? Is there a short stem or long one? – fmw42 Apr 20 '21 at 17:39
I just posted it, There is a long stem, but the depending on the sample the leaf could be wider or longer (check the images i attached). Thanks – M. Villanueva Apr 20 '21 at 17:45
My assumption was that the leaf was longer than wide for the ellipse major axis to be "along" the stem. If not, then you need to add or subtract 90 deg. For the angle, it likely needs correction according to what was posted in the answer at https://stackoverflow.com/questions/62698756/opencv-calculating-orientation-angle-of-major-and-minor-axis-of-ellipse. – fmw42 Apr 20 '21 at 17:53
Do you always have photographs with the leaf aligned either left-right or top-bottom? That is none of them at about 45 deg angle? – fmw42 Apr 20 '21 at 17:56
At the moment yes they are placed at 0, 90, 180, 270. – M. Villanueva Apr 20 '21 at 18:08
See my ADDITION in my answer above. I have added code to rotate the image so the stem points downward. – fmw42 Apr 20 '21 at 18:29
Sorry but still not working, I have been trying to use the 4 edges of the line of the elipse to try to determine where is the stem (suming all values of the regions at these edges, these regions are determined by a 100 px square with the center being each edge of the ellipse) and rotate it using arccos (using the center of the stem, the center of the image and the 180º as reference) or atan2 formulas. but im not getting the result expected yet, mostly because some solutions work well when the stem is placed at one side of the image but fail to rotate well when the stem is on the other side. – M. Villanueva Apr 22 '21 at 07:26
I do not understand. Post a new question with more details and image examples. – fmw42 Apr 22 '21 at 16:32

how can I detect a certain shape (plant stem) in a binary image?

EDIT

1 Answers1