5

I'm looking for a algorithm that computes the following: I have an image with a predefined area (the green one on the attached image). The user draws the red rectangle and the algorithm should compute whether the red rectangle matches approximately the green one. For example the position of the red rectangle on the attached picture would be ok.

What is a good way to compute this? Is there any best practice algorithm?

My idea is to compute the middle of the red rectangle and then to determine whether the middle is inside the green rectangle. In addition, I would calculate if the length and height match approximately the length and height of the green one (25% more or less).

Is this a good idea? Any other suggestion?

enter image description here

Rose Nettoyeur
  • 885
  • 1
  • 16
  • 29
  • 1
    It depends on what you are trying to achieve. – SergeyS Sep 15 '16 at 08:10
  • Is always the user drawn rectangle should overlap the image, or the rectangle can be drawn any where in the area? – Clement Amarnath Sep 15 '16 at 08:12
  • The user should select a special item on the image and the algorithm should compute whether he has selected the right item. – Rose Nettoyeur Sep 15 '16 at 08:14
  • The red rectangle can be drawn all over the image. But the algorithm should only output true when the red one matches approximately the green one. – Rose Nettoyeur Sep 15 '16 at 08:16
  • 1
    @RoseNettoyeur You need to define "matches approximately". It really depends on what is your approximation limits to be sufficient. – A. Sarid Sep 15 '16 at 08:54
  • Maybe you really have a *set of* predefined green rectangles and a user-given red rectangle, and you want to find *which* green rectangle they meant? If not, then I'd simply decide an allowable (absolute or relative; your choice) error for each of the each of the four numbers defining the top-left and bottom-right corners of the red rectangle. – j_random_hacker Sep 15 '16 at 09:20
  • What does approximate mean? The area of the rectangle or the dimension of the rectangle? – Amber Beriwal Oct 03 '16 at 11:40

3 Answers3

12

Compute the area of the intersection and divide by the average of the areas of the two rectangles (arithmetic or geometric). You will get a fraction. The closer to 1, the better the match.

  • You'd better use the union of areas instead of the average of the areas. This is called intersection over union or Jaccard index. https://en.wikipedia.org/wiki/Jaccard_index – Ismael EL ATIFI Sep 03 '19 at 10:39
  • @IsmaelELATIFI: why ? –  Sep 03 '19 at 11:46
  • Because using the average is more optimistic (gives a higher value) than using the union. And the IoU score (Intersection over Union) is more interpretable/intuitive. For example, imagine you have a square of area 10 completely inside a square of area 100. In this case, the IoU is 10/100 = 10% which is more intuitive than 10/((10+100)/2) = 10/55 = 18.2% given with your formula. – Ismael EL ATIFI Sep 04 '19 at 12:16
  • @IsmaelELATIFI: very subjective. –  Sep 04 '19 at 12:27
  • So the resulting number, lets call it "scale", what do we do with it? Do we multiply one rectangle width/height by that number to match it to other one ? Whats the point of knowing that its A is 10% of B. Does it mean its b.width()*scale & b.height*scale? – Dariusz Aug 23 '22 at 09:58
  • @Dariusz: call it "matching fraction". –  Aug 27 '22 at 18:52
1
  • Take the average distance between vertices as the criteria for mismatch.
  • Lets assume first rectangle's vertices are [x1,y1], [x2,y2], [x3,y3], [x4,y4] and for second one are [a1,b1],[a2,b2],[a3,b3],[a4,b4]
  • Get euclidiean distance between these points
  • Lower distance means better match, e.g exact overlap will give 0, a shape shift or offset shift of any rectangle would increase the average distance of vertices.

enter image description here

DhruvPathak
  • 42,059
  • 16
  • 116
  • 175
1

Investigating the problem, I tend to think about the conditions that should make the comparison of the green and the red rectangles fail, together with reasoning about the failing conditions, separately about each condition.

What I mean above, practically, is that I would like the following responses from the algorithm, making clear what aspect of the comparison fails:

  1. Your rectangle's width is way off.
  2. Your rectangle's height is way off.
  3. Your rectangle's horizontal placement is way off.
  4. Your rectangle's vertical placement is way off.

Let us call the conditions above "failing conditions". These failing conditions suggest my view of the comparison, which unavoidably directs my approach. One could view it differently ("Your rectangle's area is way off."). The user, of course, could get more generic responses like the following:

  • Your rectangle's dimensions are way off.
  • Your rectangle's placement is way off.
  • Your rectangle is way off. Try again.
  • Dude, are you drunk?

In the following I use green to refer to the green rectangle as an object and red to refer to the red rectangle as an object. All conditions are based on relative errors, that is absolute errors normalized with respect to the actual values, i.e. the values of the green rectangle.

One thing that needs to be specified is what "way off" means for horizontal and vertical placement. It means that there is a divergence between the location of a key point of the green rectangle and the location of the corresponding key point of the red rectangle. Let us choose the center of a rectangle as the key point for comparisons (one could choose the top-left corner of the rectangle).

Another thing that needs to be specified is how you may compare two points in a relative way, separately for each axis. You need a reference value. What you can do is calculate the absolute offset between the two points in each axis. Then you can calculate the relative offset with respect to the green rectangle's corresponding dimension. For instance, you can calculate the relative horizontal offset as the absolute offset between the centers in the x-axis divided by the width of the green rectangle. All in all, for a comparison to succeed, I would like the rectangles to have almost the same dimensions and almost the same center. Where "almost" should be quantified as a percentage.

Concerning failing condition (1), assuming that the maximum allowed relative error for the rectangle's width is 25%, the boolean value that we have to calculate is:

| green.width - red.width | / green.width > 0.25

If the value above is true, then failing condition (1) goes off. The Dude may be drunk. We may exit and notify.

Concerning failing condition (2), assuming that the maximum allowed relative error for the rectangle's height is 30%, the boolean value that we have to calculate is:

| green.height - red.height | / green.height > 0.30

If the value above is true, then failing condition (2) goes off. We may exit and notify.

Concerning failing condition (3), assuming that the maximum allowed relative error for the rectangle's horizontal offset is 15%, the boolean value that we have to calculate is:

| green.center.x - red.center.x | / green.width > 0.15

If the value above is true, then failing condition (3) goes off. We may exit and notify.

Concerning failing condition (4), assuming that the maximum allowed relative error for the rectangle's vertical offset is 20%, the boolean value that we have to calculate is:

| green.center.y - red.center.y | / green.height > 0.20

If the value above is true, then failing condition (4) goes off. We may exit and notify.

If at least one failing condition goes off, then the comparison fails. If no failing condition is true, then the comparison is successful, the green and the red rectangles are almost the same.

I believe that the approach above has a lot of advantages, such as reasoning for separate aspects of the comparison, as well as defining different thresholds for the failing conditions. You can also tune the thresholds according to your taste. In extreme cases more parameters may need to be taken into account, though.

xnakos
  • 9,870
  • 6
  • 27
  • 33