Detecting near-horizontal lines using Image processing

Question

Is there any way to use opencv to detect lines that are nearly horizontal? I've muddled my way through some of the concepts mentioned in How to detect lines in OpenCV? -- I can get edge detection with canny, but I'm kind of lost on how to use Hough transforms and constrain them to horizontal lines.

I have a bunch of example images here: https://gist.github.com/jason-s/df90e41e29f3ba46e6ccabad4516e916

including:

In particular each image has a pair of horizontal edges that are approximately 1200 pixels long and within 3 degrees of horizontal. (These are formed by corners of photographs I scanned in.)

Any suggestions on what algorithm to use?

Iteratively rotate the image. Then get the average of each row and find the row with the largest value. Find the row with the largest value over all the rotation angles. — fmw42, Dec 24 '21 at 22:13
potentially related: https://stackoverflow.com/questions/70476326/opencv-photograph-extraction-from-scans — Christoph Rackwitz, Dec 24 '21 at 22:21
just do a hough transform for lines, and then filter the resulting list. no there is no "function" for that. I mean writing code, like a loop or list comprehension. — Christoph Rackwitz, Dec 24 '21 at 22:22
OK. I know how to filter with a loop or list comprehension... what I don't know is whether to use HoughLines or HoughLinesP, and how to make the best use of them. Is there any way to get some kind of confidence level of the lines it finds? — Jason S, Dec 24 '21 at 22:41
What is your end goal? Why do you need to find these lines? Are you trying to extract the images? — fmw42, Dec 24 '21 at 22:43
These are from some images I scanned where I have four 4x6" photographs on each one, and I'm trying to find the corners so I can extract them. I've looked for more general rectangle-finding software and it seems to have trouble. But I know almost exactly where the corners are, I just have to correct for misalignment. — Jason S, Dec 24 '21 at 22:51
@fmw42 "find the row with the largest value" ???? `HoughLines()` doesn't give you a largest value, it just gives R and theta of lines it finds. — Jason S, Dec 24 '21 at 22:53
Check out this approach, it might be useful: https://stackoverflow.com/questions/67644977/recognizing-corners-page-with-opencv-partialy-fails/67645614#67645614 — stateMachine, Dec 24 '21 at 23:18
@stateMachine I'll try but these are color photographs without any guarantee of contrast against the surrounding white background. (I mean they're not all white or even mostly white, but I'm not sure how well this will work....) — Jason S, Dec 24 '21 at 23:25
@Jason S I was not suggesting they are related. I was just giving you a method to find the gaps. As to extracting the pictures, I would approach it by thresholding or better flood filling. Then using contours to find the picture regions in the binary image. Then from the contours, get the rotated rectangles using minAreaRect(). — fmw42, Dec 24 '21 at 23:25
Of course that means that your background should contrast against the edges of your image so that the flood fill will work (or at least be a constant color not along the edge). The edge does not have to be perfect, just not too large gaps from the flood fill "eating" into the image — fmw42, Dec 25 '21 at 00:18
Maybe you should include the full images, so we can check them out and propose alternate ways of detecting the corners. — stateMachine, Dec 25 '21 at 00:40
I think I have enough to go on with `HoughLines`. I'd have to blur out the center of the images. — Jason S, Dec 25 '21 at 00:46

Mario Abbruscato · Answer 1 · 2021-12-25T04:03:42.417

Lines detection an filter by line degree of orientation

import cv2
import numpy as np
import math

path='images/lines.png'
image = cv2.imread(path)

dst = cv2.Canny(image, 50, 200, None, 3)
linesP = cv2.HoughLinesP(dst, 1, np.pi / 180, 50, None, 50, 10)

if linesP is not None:
    for i in range(0, len(linesP)):
        l = linesP[i][0]

        #here l contains x1,y1,x2,y2  of your line
        #so you can compute the orientation of the line 
        p1 = np.array([l[0],l[1]])
        p2 = np.array([l[2],l[3]])

        p0 = np.subtract( p1,p1 ) #not used
        p3 = np.subtract( p2,p1 ) #translate p2 by p1

        angle_radiants = math.atan2(p3[1],p3[0])
        angle_degree = angle_radiants * 180 / math.pi

        print("line degree", angle_degree)

        if 0 < angle_degree < 15 or 0 > angle_degree > -15 :
            cv2.line(image,  (l[0], l[1]), (l[2], l[3]), (0,0,255), 1, cv2.LINE_AA)


cv2.imshow("Source", image)

print("Press any key to close")
cv2.waitKey(0)
cv2.destroyAllWindows()

score 2 · Answer 2 · answered Dec 30 '21 at 16:13

I had an attempt at this using a variation on Fred's (@fmw42) suggestion. I first tried to locate all the white pixels along the image border, by converting to HSV colourspace then finding pixels that are both unsaturated and bright.

I then rotated the resulting image through -5 to +5 degrees in 0.1 degree increments. At each angle of rotation, I ran a SobelY filter looking for horizontal edges. Then I counted the white pixels in each row. Any time I find an orientation that results in a longer horizontal line, I update my best estimate and remember the rotation.

There are many variations possible but this should get you started:

#!/usr/bin/env python3

import cv2
import numpy as np

# Load image
im = cv2.imread('a4e.jpg')

# Find white pixels, i.e. unsaturated and bright
HSV = cv2.cvtColor(im, cv2.COLOR_BGR2HSV)

unsat = HSV[:,:,1] < 50
bright= HSV[:,:,2] > 240

white = ((unsat & bright)*255).astype(np.uint8)
cv2.imwrite('DEBUG-white.png', white)

That looks like this:

# Pad with border so it isn't cropped when rotated, get new dimensions
bw = 100
white = cv2.copyMakeBorder(white, bw, bw, bw, bw, borderType= cv2.BORDER_CONSTANT)
w, h = white.shape[:2]

# Find rotation that results in horizontal row with largest number of white pixels
maxOverall = 0

# SobelY horizontal edge kernel
kernel = np.array((
    [-1, -2, -1],
    [0, 0, 0],
    [1, 2, 1]), dtype="int")

# Rotate image -5 to +5 degrees in 0.1 degree increments
for angle in [x * 0.1 for x in range(-50, 50)]:
   M = cv2.getRotationMatrix2D((h/2,w/2),angle,1)
   rotated = cv2.warpAffine(white,M,(h,w))
   # Output image for debug purposes
   cv2.imwrite(f'DEBUG rotated {angle}.jpg',rotated)

   # Filter for horizontal edges
   filtered = cv2.filter2D(rotated, -1, kernel)
   cv2.imwrite(f'DEBUG rotated {angle} filtered.jpg',filtered)

   # Check for maximum white pixels in any row
   maxThis = np.amax(np.sum(rotated, axis=1))
   if maxThis > maxOverall:
      print(f'Angle:{angle}: New longest horizontal row has {maxThis} white pixels')
      maxOverall = maxThis

The overall process looks like this:

The output looks like this, which means the detected angle is 0.6 degrees:

Angle:-5.0: New longest horizontal row has 34287 white pixels
Angle:-4.9: New longest horizontal row has 34517 white pixels
Angle:-4.800000000000001: New longest horizontal row has 34809 white pixels
Angle:-4.7: New longest horizontal row has 35191 white pixels
Angle:-4.6000000000000005: New longest horizontal row has 35625 white pixels
Angle:-4.5: New longest horizontal row has 36108 white pixels
Angle:-4.4: New longest horizontal row has 36755 white pixels
Angle:-4.3: New longest horizontal row has 37436 white pixels
Angle:-4.2: New longest horizontal row has 38151 white pixels
Angle:-4.1000000000000005: New longest horizontal row has 38876 white pixels
Angle:-4.0: New longest horizontal row has 39634 white pixels
Angle:-3.9000000000000004: New longest horizontal row has 40414 white pixels
Angle:-3.8000000000000003: New longest horizontal row has 41240 white pixels
Angle:-3.7: New longest horizontal row has 42074 white pixels
Angle:-3.6: New longest horizontal row has 42889 white pixels
Angle:-3.5: New longest horizontal row has 43570 white pixels
Angle:-3.4000000000000004: New longest horizontal row has 44252 white pixels
Angle:-3.3000000000000003: New longest horizontal row has 44902 white pixels
Angle:-3.2: New longest horizontal row has 45776 white pixels
Angle:-3.1: New longest horizontal row has 46620 white pixels
Angle:-3.0: New longest horizontal row has 47414 white pixels
Angle:-2.9000000000000004: New longest horizontal row has 48178 white pixels
Angle:-2.8000000000000003: New longest horizontal row has 48705 white pixels
Angle:-2.7: New longest horizontal row has 49225 white pixels
Angle:-2.6: New longest horizontal row has 49962 white pixels
Angle:-2.5: New longest horizontal row has 51501 white pixels
Angle:-2.4000000000000004: New longest horizontal row has 53217 white pixels
Angle:-2.3000000000000003: New longest horizontal row has 54997 white pixels
Angle:-2.2: New longest horizontal row has 56926 white pixels
Angle:-2.1: New longest horizontal row has 59033 white pixels
Angle:-2.0: New longest horizontal row has 61017 white pixels
Angle:-1.9000000000000001: New longest horizontal row has 62538 white pixels
Angle:-1.8: New longest horizontal row has 63370 white pixels
Angle:-1.7000000000000002: New longest horizontal row has 64144 white pixels
Angle:-1.6: New longest horizontal row has 65685 white pixels
Angle:-1.5: New longest horizontal row has 68510 white pixels
Angle:-1.4000000000000001: New longest horizontal row has 72377 white pixels
Angle:-1.3: New longest horizontal row has 76693 white pixels
Angle:-1.2000000000000002: New longest horizontal row has 80932 white pixels
Angle:-1.1: New longest horizontal row has 84101 white pixels
Angle:-1.0: New longest horizontal row has 86557 white pixels
Angle:-0.9: New longest horizontal row has 90499 white pixels
Angle:-0.8: New longest horizontal row has 97179 white pixels
Angle:-0.7000000000000001: New longest horizontal row has 101430 white pixels
Angle:-0.6000000000000001: New longest horizontal row has 105001 white pixels
Angle:-0.5: New longest horizontal row has 112976 white pixels
Angle:-0.4: New longest horizontal row has 117256 white pixels
Angle:-0.30000000000000004: New longest horizontal row has 131478 white pixels
Angle:-0.2: New longest horizontal row has 141468 white pixels
Angle:-0.1: New longest horizontal row has 164588 white pixels
Angle:0.0: New longest horizontal row has 186150 white pixels
Angle:0.1: New longest horizontal row has 206695 white pixels
Angle:0.2: New longest horizontal row has 230821 white pixels
Angle:0.30000000000000004: New longest horizontal row has 249003 white pixels
Angle:0.4: New longest horizontal row has 258888 white pixels
Angle:0.6000000000000001: New longest horizontal row has 264409 white pixels

score 0 · Answer 3 · answered Dec 25 '21 at 23:55

0

You can find the angle by which the line is parallel to ground by using tan inverse.

for x1,y1,x2,y2 in lines[0]:
   angle = math.degrees(math.atan((abs(y2-y1))/abs(x2-x1)))
   cv2.line(img2,(x1,y1),(x2,y2),(255,0,0),1)
   print(angle)

Then you can filter the lines as told by @Mario. Since above uses abs() to find difference, you will have to filter the angles only using a positive range.

answered Dec 25 '21 at 23:55

Demnofocus

21
6

thanks, I don't need to use atan though; you can check whether the slope is within 3 degrees by precomputing tan 3 ~ 0.0524. – Jason S Dec 26 '21 at 05:21
Great, yeah you could use the value of tan 3 itself too. – Demnofocus Dec 27 '21 at 05:13

Detecting near-horizontal lines using Image processing

3 Answers3

Linked