3

I'm looking for a way to split a number of images into proper rectangles. These rectangles are ideally shaped such that each of them take on the largest possible size without containing a lot of white.

So let's say that we have the following image input

I would like to get an output such as this: output

Note the overlapping rectangles, the hole and the non axis aligned rectangle, all of these are likely scenario's I have to deal with.

I'm aiming to get the coordinates describing the corner pieces of the rectangles so something like

[[(73,13),(269,13),(269,47)(73,47)],
 [(73,13),(73,210),(109,210),(109,13)]
...]

In order to do this I have already looked at the cv2.findContours but I couldn't get it to work with overlapping rectangles (though I could use the hierarchy model to deal with holes as that causes the contours to be merged into one.

Note that although not shown holes can be nested.

Thijser
  • 2,625
  • 1
  • 36
  • 71
  • If anybody has any extra questions please ask them, partial answers are also helpful as is idle speculation. – Thijser Jul 27 '20 at 14:32
  • If you can find end points of parallel line segments after simplifying contours by [`approxPolyDP`](https://docs.opencv.org/2.4/modules/imgproc/doc/structural_analysis_and_shape_descriptors.html#approxpolydp), you can take the longer segment and reflect it to the other in order to get the corner points. – Burak Jul 27 '20 at 18:09
  • @Burak isn't that going to cause trouble if you have holes in the shape? – Thijser Jul 28 '20 at 06:21
  • Yes, holes should be checked too. If there is a hole at short segment level, that rectangle is not valid. If there is a hole not at the short one but at the long segment level, there remains a valid rectangle after throwing the part with hole out. – Burak Jul 28 '20 at 09:23
  • You might luckier asking mathematicians on the Mathematics Stack Exchange site. – Mark Setchell Aug 07 '20 at 10:26

1 Answers1

2

A algorithm that works roughly as follow should be able to give you the result you seek.

  1. Get all the corner points in the image.
  2. Randomly select 3 points to create a rectangle
  3. Count the ratio of yellow pixels within the rectangle, accept if the ratio satisfy a threshold.
  4. Repeat 2 to 4 until : a) every single combination of point is complete or b) all yellow pixel are accounted for or c) after n number of iteration

The difficult part of this algorithm lies in step 2, creating rectangle from 3 points.

If all the rectangles were right angle, you can simply find the minimum x and y to correspond for topLeft corner and maximum x and y to correspond for bottomRight corner of your new rectangle.

But since you have off axis rectangle, you will need to check if the two vector created from the 3 points have a 90 degree angle between them before generating the rectangle.

yapws87
  • 1,809
  • 7
  • 16
  • Interesting idea, maybe it will work with Hough Transform to get the corner points and findcontours to get the general shapes. Maybe something with minAreaRect for the 3 points and then muliply the result with a mask to see how far from the threshold we are. – Thijser Jul 28 '20 at 07:41
  • 1
    minAreaRect() is the best function to use for step 2! – yapws87 Jul 29 '20 at 05:40