0

I want to see in Python which direction arrows point in pictures. In the following algorithm I calculate the center of gravity of the arrows:

image = cv2.imread(image.tiff")
gray = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
blurred = cv2.GaussianBlur(gray, (5, 5), 0)
thresh = cv2.threshold(blurred, 60, 255, cv2.THRESH_BINARY)[1]

cnts = cv2.findContours(thresh.copy(), cv2.RETR_CCOMP, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if imutils.is_cv2() else cnts[1]

for c in cnts:
    M = cv2.moments(c)
    cX = int(M["m10"] / M["m00"])   # center of gravity (x coordinate)
    cY = int(M["m01"] / M["m00"])   # center of gravity (y coordinate)

enter image description here (the red point is the center of gravity)

Is there another way to determine the center point of the arrows so that the difference between the center point and the center of gravity can then be used to calculate the direction in which the arrow points?

Jeru Luke
  • 20,118
  • 13
  • 80
  • 87
freddykrueger
  • 309
  • 4
  • 17
  • [Potentially of use](https://stackoverflow.com/questions/22876351/opencv-2-4-8-python-determining-orientation-of-an-arrow) – GPPK Mar 02 '18 at 11:47
  • Cant you just try to plot a trend line on this data? (as in, transform the 2d image to a 1d one, where the location (and value) are used in the appropriate way. The fun here is that you can figure out on how to do this the `best' way). Because then the slope of this trendline already has your answer. – zwep Mar 02 '18 at 11:56
  • @GPPK: The problem is that the arrows don't always look so clean. – freddykrueger Mar 02 '18 at 13:34
  • @zwep: Could you elaborate on that, please? I'm not quite going to get into it yet. Would it also be possible to calculate the center of the arrow from the extreme points? (as done here: https://www.pyimagesearch.com/2016/04/11/finding-extreme-points-in-contours-with-opencv/) – freddykrueger Mar 02 '18 at 13:34
  • @freddykrueger Did my answer help you? Or have you found a different solution? – zwep Mar 04 '18 at 13:29
  • @zwep The problem is that I want to detect the orientation of street arrows of digital orthophotos so the resolution is not very high and there are other arrows, like the example in the question (right arrow, left arrow, straight, left arrow ...) – freddykrueger Mar 05 '18 at 07:48
  • Ah I see, so do you also face issues with detecting the arrows themselves? As in.. given a picture with X arrows, can you extract those X arrows? – zwep Mar 05 '18 at 08:20
  • I have created my own algorithm, which first calculates the center of the arrows, then erode the arrows (using skimage.morphology.erosion) and then calculates the center of gravity of the largest contour left. Then the direction of the arrow can be calculated by sinus... – freddykrueger Mar 06 '18 at 13:05

1 Answers1

1

Here my more elaborate answer... Hopefully the comments are clear enough!

import matplotlib.pyplot as plt
import imageio
import os
import numpy as np

# Load the image with imageio...
base_path = r'D:\temp'
im_name = 'arrow_image.png'
im_dir = os.path.join(base_path, im_name)
A = imageio.imread(im_dir)

# Get all the points that have a value in the A[:,:,0]-dimension of the image
pts2 = np.array([[j,i] for i in range(A.shape[0]) for j in range(A.shape[1]) if A[i,j,0] > 0])
# Using convexhull.. get an overlay of vertices for these found points
ch = ConvexHull(pts2)

# If you want to visualize them.. you can do that by
plt.plot(pts2[:, 0], pts2[:, 1], 'ko', markersize=10)
plt.show()

# Be aware that in this whole process your image might look like it has rotated.. so be aware of the dimension (and the way stuff is visualized)

# Get the indices of the hull points.
hull_indices = ch.vertices

# These are the actual points.
hull_pts = pts2[hull_indices, :]

# With this you are able to determine the 'end' points of your arrow. Fiddle around with it, know what you are trying to get and understand it.
z = [np.linalg.norm(x) for x in pts2]
z_max_norm = np.where(z==max(z))
z_min_norm = np.where(z==min(z))
y_max = np.where(pts2[:,1] == min(pts2[:,1]))[0]
x_min = np.where(pts2[:,0] == min(pts2[:,0]))[0]

# And here some more visualization tricks
plt.fill(hull_pts[:,0], hull_pts[:,1], fill=False, edgecolor='b')
# plt.scatter(pts2[z_min,0],pts2[z_min, 1])
# plt.scatter(pts2[z_max,0],pts2[z_max, 1])
plt.scatter(pts2[y_max,0],pts2[y_max, 1])
plt.scatter(pts2[x_min,0],pts2[x_min, 1])

plt.show()

This uses the same strategy that is posed in the answer of @GPPK. My other solution was a transformation of the data to 1d... you can think of transformations like

pts3 = np.array([A[i,j,0]*i*np.log(j) for i in range(A.shape[0]) for j in range(A.shape[1]) if A[i,j,0] > 0])
plt.plot(pts3)
plt.show()

But you have to test this a little bit, maybe change some of the parameters. The idea is that you discount certain dimensions in order to get back the shape of the arrow.

zwep
  • 1,207
  • 12
  • 26