1

I'd like to find the corners of the following box

box

However, as you can see I'm detecting a lot of corners I don't want to find. I'm completly stuck on this one. No matter what I try, I always seem to find corners in the dots on the box. I used the function goodFeaturesToTrack() but I also tried cornerHarris()

The most important thing to me is to find the coordinates of the corner pixels so I can draw a wire frame.

Kind regards, Schweini

Edit: original picute

Schweini14
  • 23
  • 5
  • 1
    those *are* corners. 2D corners, visible corners. these algorithms don't see 3D. you're asking for approaches to recover the 3D pose of a completely black block, of which you need to know the dimensions beforehand. be prepared to require AI for this. for robust computer vision/machine vision, one would modify the object to have easily detectable features (AR markers on its faces, for example). – Christoph Rackwitz Apr 17 '22 at 12:36
  • Im affraid the dots on the box is the best I can do. This is a school assignment so I can't add anny AR markers I'm affraid. I do have the dimensions: width 86mm, lenght 160mm, depth 53mm – Schweini14 Apr 19 '22 at 09:00
  • unless the school project *forbids* you from sticking things on the box, you can just print a few aruco markers and put them on the box https://chev.me/arucogen/ opencv comes with aruco and a bunch of examples (drawing of cubes on top of markers, can be generalized to match your box) -- if your task is *not* to find the position of that specific box, but to literally find the corners of unknown textureless boxes, that requires some level of machine intelligence above simple Augmented Reality – Christoph Rackwitz Apr 19 '22 at 10:27
  • I'm guessing those colored blobs on the box are meant to give you "corners" (points) to work with... so you have to detect them *by hue* with color space transformation and inRange... however, then you'll have to define a model of those points on the box (model = list of 3d points) and use `solvePnP`. that's about the same thing aruco does, but aruco hides that stuff so you don't have to deal with it. – Christoph Rackwitz Apr 19 '22 at 10:48
  • Alright. Thats my first step I believe, model the 3D points. I have tried to find a homography between two images but I don't believe that's a suitable way since the camera angles are all slightly different – Schweini14 Apr 20 '22 at 09:49
  • I'd like a picture of the box with no annotations on it, for experimentation. preferably a close-up shot. I think I can demonstrate the "straight segments from contour" step at least. – Christoph Rackwitz Apr 20 '22 at 10:02
  • Sure I added it in the original question – Schweini14 Apr 20 '22 at 11:23

1 Answers1

2

To draw the wire frame onto the image, following process can be thinkable.

When extracting outline of the black box region, the outline consists with 6 straight line segments.
Therefore, you'll able to find at least 6 corners of the box as the intersection of each two adjacent line segments.

Additionally, it looks like that, outline of 6 surfaces of the box will be able to coarsely estimated from each adjacent line segment pairs. (with assuming parallelogram)
This means estimating the positions of the remaining two corners (to draw the wire frame).

Furthermore, if you want, comparing the estimation result with your examination (corner detecition) result will be able to refine the coordinates of one corner.

fana
  • 1,370
  • 2
  • 7
  • Because 1 corner of the box is hidden (occluded with the box itself), it can't be detected directly. So, to draw wireframe, you need to use some "estimate". – fana Apr 20 '22 at 04:48
  • Verry true. My real problem is finding the corners of the box. Once I have those coordinates it should be easy. I've been trying to mask the box and find the corners in the contours. However, opencv does not determine the same corners in real life to be the most significant corner in order making this method a pain. – Schweini14 Apr 20 '22 at 09:30
  • If you try the process written in this answer, you don't have to start with troublesome 2D corner detection (such as goodFeatruesToTrack(), cornerHarris(), etc). What you will try at first is just find the region of the box (e.g. binalize and then contour detection). If you obtainen the box region well, it means that outline of the box region extracted. – fana Apr 20 '22 at 09:51
  • In your image, the background is white and the box is black, so it looks like that this 1st step shouldn't be too difficult. – fana Apr 20 '22 at 09:55
  • I like this approach. it's simple. sadly, opencv lacks some functions to deal with contours that would make getting straight segments a lot easier (cut contour up into straight and curved segments). you'll have to mess around with hough transform instead. -- given the 6 sides, you can have 6 of 8 corners, and the two missing ones are always opposite (one occluded, one pointing at you, "visible" but not part of outline). you can run through the four possible cases, selecting the 6 corresponding model points for a trial in each case, and see how that fits (solvePnP) – Christoph Rackwitz Apr 20 '22 at 09:59
  • approxPolyDP may have its uses but here it's very useless... except to determine what contour points are corners, and then to split the original contour at _those_ points, cut back a bit (your box corners are rounded a bit), and then you'd have straight line segments from the contour. – Christoph Rackwitz Apr 20 '22 at 10:01
  • With this image, when box region detection result mask (as binary image) is cleary obtained (perfect shape)... Blur it, and checking difference between the blurred and un-blurred mask along the outline of region will become help to extract the line segments, I think. – fana Apr 20 '22 at 10:25