Getting points in 2D plane under perspective by real distance using Aruco marker

Question

I have ruler which is included on the picture below. The dimensions of the marker, color checker and space between the marker and the color checker are known.

The problem is how to compute position of the color checker's corners (C1-C4) from the coordinates of the aruco marker (M1-M4) when the image is taken under perspective so just multiplying vectors (M2-M1) and (M3-M2) by scalar computed from marker dimensions and color checker dimensions is not possible.

The implementation is in C++ using OpenCV but code is not necessary.

Thanks in advance

Edit: I tried to use perspective transform

const std::vector<cv::Point2f> ruler_inner_rect_real = {
    left_marker.at(0),
    left_marker.at(1),
    left_marker.at(2),
    left_marker.at(3)
};
const std::vector<cv::Point2f> RULER_INNER_RECT_IDEAL = {
    cv::Point2f(0, 0),
    cv::Point2f(this->marker_size, 0),
    cv::Point2f(this->marker_size, this->marker_size),
    cv::Point2f(0, this->marker_size),
};
cv::Mat tmat = cv::getPerspectiveTransform(ruler_inner_rect_real, RULER_INNER_RECT_IDEAL);

But this thing doesn't work because the RULER_INNER_RECT_IDEAL is in the zero position of the image/Mat. What are the correct coordinates to be used with the perspective transform?

Edit: So with the help in comments I got the solution to get points C1-C4:

const std::vector<cv::Point2f> in = {
    cv::Point2f(this->marker_size+this->color_checker_position_mm.width, this->color_checker_position_mm.height),
    cv::Point2f(this->marker_size+this->color_checker_position_mm.width+this->color_checker_width, this->color_checker_position_mm.height),
    cv::Point2f(this->marker_size+this->color_checker_position_mm.width+this->color_checker_width, this->marker_size+this->color_checker_position_mm.height),
    cv::Point2f(this->marker_size+this->color_checker_position_mm.width, this->marker_size+this->color_checker_position_mm.height),
};
std::vector<cv::Point2f> out;
cv::perspectiveTransform(in, out, tmat.inv());

it's all just transformation matrices, transforming from one frame to another. rvec/tvec are encodings, not usable for composition. you need a translation in marker space. then you need to compose that with the marker pose (transforms into camera space, from marker space), in the right way. — Christoph Rackwitz, Jul 26 '23 at 08:06
@ChristophRackwitz I edited question - added what I tried with the perspective transform. — Matej, Jul 26 '23 at 08:31

score 0 · Accepted Answer · answered Jul 26 '23 at 08:16

0

You know "paper" coordinates of all rectangle corners, coordinates of black rect corners at the image and want to find color rect image coordinates?

In this case you can use OpenCV getPerspectiveTransform function to calculate perspective transformation matrix for black rect (and for whole image), then apply perspectiveTransform to transform color rect corners.

answered Jul 26 '23 at 08:16

MBo

77,366
5
53
86

Yes, I was thinging about that but I don't actually know the coordinates of the marker in other space, do I? – Matej Jul 26 '23 at 08:27
I added what I tried into the question. – Matej Jul 26 '23 at 08:31
Make grayscale, choose [threshold](https://docs.opencv.org/4.x/d7/d4d/tutorial_py_thresholding.html) to exclude gray(former color), so only black will present at the image, then [look for corners](https://docs.opencv.org/3.4/dc/d0d/tutorial_py_features_harris.html) – MBo Jul 26 '23 at 08:35
No, you got me wrong. I know coordinates of the marker in the image but not in the second space. Or I do but not really :D What is the correct RULER_INNER_RECT_IDEAL from the question? – Matej Jul 26 '23 at 08:38
So you need to tie "paper" coordinates to the ruler? – MBo Jul 26 '23 at 08:43
I know the image coordinates of the marker, real life dimensions of the marker, color checker and distance between these two but I need the image coordinates of the color checker from that – Matej Jul 26 '23 at 08:44
I had re-read your question. Method should work, but for getting C1'-C4' at the image you have to calculate inverse matrix: `getPerspectiveTransform(RULER_INNER_RECT_IDEAL,ruler_inner_rect_real)` – MBo Jul 26 '23 at 08:56
Ahh, I was so focused to the vectors and scaling them that I completely forgot about just transforming real life points. I put solution to the question. Thank you for your help! – Matej Jul 26 '23 at 09:09

Getting points in 2D plane under perspective by real distance using Aruco marker

1 Answers1