find orientation of pixel patch in affine transformed image with respect to original reference

Question

In part of my project I need to compute orientation of a patch in an affine transformed image. Now my problem is that I don't know how can I find this computed orientation with respect to original un-warped image.

For example a point in warped image is found(100,200). I can extract orientation of this point using 8x8 neighboring pixels. suppose it is 30 degree. the warped image in which the point has found is result of applying transformation on original image with 60 degree in each axis.(pitch,yaw and roll).(This orientation extraction is usually known as descriptor extraction in computer vision)

Now the transformation matrix is known. orientation of the point in the transformed image is known as well. The position of the point wrt reference is known(using inverse transformation). Now I want to know what is the new orientation if this point(100,200) goes to reference frame(e.g. 150,250). in another word, what is the new orientation with respect to reference image.

I know this is straight forward to solve if we just have roll angle rotation(60 degree). in this case the orientation wrt reference frame would be 30+60 = 90 .

I tried implement this using OpenCV:

        cv::Mat rvec1(3,1,CV_32F); // rot vector related to B
        rvec1.at<float>(0,0)=0;
        rvec1.at<float>(1,0)=30*to_RAD;
        rvec1.at<float>(2,0)=0;

        cv::Mat rvec2(3,1,CV_32F); // rot vector related to A
        rvec2.at<float>(0,0)=0;
        rvec2.at<float>(1,0)=60*to_RAD;
        rvec2.at<float>(2,0)=0;

        cv::Mat R_A;
        cv::Mat R_B;

        cv::Rodrigues(rvec1, R_B);
        cv::Rodrigues(rvec2, R_A);

        cv::Mat R_combined= R_B*R_A;

        cv::Mat rvec_result;
        cv::Rodrigues(R_combined,rvec_result);

I want to create rotation Mat A and B using 2 rotation vector. and after multiplying these two I want to convert it into rotation vector.But only thing I get is a runtime error on the last line (cv::Rodrigues(R_combined,rvec_result);)

Thank you for your help in advance.

The simplest way to do that is to identify at least 3 points in both images of the patch and then find the affine transform that relates those points. Are you able to detect points consistently in this patch? — Hammer, Sep 05 '12 at 15:07
The problem is not about finding transformation. the transformation is known. I found the original position with inverse transformation Matrix. but I don't know how can I compute orientation of a point in warped image with respect to original image.suppose we rotate original image by angle theta. now the orientation of a point in rotated image wrt original image is orientation+theta . so what about transforming with three angles pitch,yaw and roll ? — batista cori, Sep 05 '12 at 15:51
wait, you know the transform? You know the matrix relating your original patch and the transformed patch? Are you just trying to extract euler angles from the matrix? — Hammer, Sep 05 '12 at 17:44
No. not euler angles. I extracted orientation of a point using its neighboring 8x8 pixels.(some kind of gradient calculation). now I want to know its new orientation in reference image. suppose we have a point with an arrow assigned to it in warped image. we want to know direction(angle) of this arrow wrt reference image. — batista cori, Sep 05 '12 at 18:30
So you have a patch, you know its orientation relative to reference, and you have an arrow, you know its orientation relative to the patch, and you want the orientation of the arrow wrt to reference? — Hammer, Sep 05 '12 at 19:03
yes. except that we don't know orientation of patch relative to reference.(it can be calculated from transformation mat). I wanted upload an image showing the problem.but am not able so due to low reputation. :( — batista cori, Sep 05 '12 at 19:49
If it can be calculated from the transformation mat, why do you say you don't know the orientation of the patch relative to reference? Are you asking how to calculate it? You should update your question with a much clearer explanation of the problem. Give an example of the numbers you have and what exactly you are trying to calculate in that example — Hammer, Sep 05 '12 at 19:57

Hammer · Accepted Answer · 2012-09-06T15:07:27.297

Ok it sounds like you have two rotation matrices (you can use rodrigues to get them) call them A and B. You want to know how to combine them. Lets say that A represents the orientation of your arrow wrt to the patch and B is the patch wrt to the origin. Lets start with our origin at the center of the patch. A describes the orientation of the arrow. Now we want to rotate the origin by B so that our original reference axis are now at B wrt to the new origin. To do that just do

Combined = [B]*[A];

Update

I don't know why your code would give you an error, it works perfectly for me. Here is the code I ran.

cv::Mat rvec1(3,1,CV_32F); // rot vector related to B
rvec1.at<float>(0,0)=0;
rvec1.at<float>(1,0)=30*M_PI/180;;
rvec1.at<float>(2,0)=0;

cv::Mat rvec2(3,1,CV_32F); // rot vector related to A
rvec2.at<float>(0,0)=0;
rvec2.at<float>(1,0)=60*M_PI/180;;
rvec2.at<float>(2,0)=0;

cv::Mat R_A;
cv::Mat R_B;

cv::Rodrigues(rvec1, R_B);
cv::Rodrigues(rvec2, R_A);

cv::Mat R_combined= R_B*R_A;

cv::Mat rvec_result;
cv::Rodrigues(R_combined,rvec_result);

std::cout << rvec1 << std::endl<<std::endl;
std::cout << rvec2 << std::endl<<std::endl;
std::cout << R_A << std::endl<<std::endl;
std::cout << R_B << std::endl<<std::endl;
std::cout << R_combined << std::endl<<std::endl;
std::cout << rvec_result << std::endl<<std::endl;

And here is my output

[0; 0.52359879; 0]

[0; 1.0471976; 0]

[0.49999997, 0, 0.86602545;
0, 1, 0;
-0.86602545, 0, 0.49999997]

[0.86602539, 0, 0.5;
0, 1, 0;
-0.5, 0, 0.86602539]

[-5.9604645e-08, 0, 1;
0, 1, 0;
-1, 0, -5.9604645e-08]

[0; 1.5707964; 0]

Yes.I think this is what I am talking about. But I'm not quite sure. I'm updating my code to see the result.I'll update my question afterward. Thanks — batista cori, Sep 05 '12 at 23:06
Hi Hammer. I tried to get result from your answer using OpenCV. But I was not able to get it. I posted my code in the question. Can you please provide me with an advise how to do it? — batista cori, Sep 06 '12 at 13:33
So strange!!! Actually I'm using Qt framework and opencv 2.4 It works for vector to mat conversion but sucks vice versa. — batista cori, Sep 06 '12 at 15:42

find orientation of pixel patch in affine transformed image with respect to original reference

1 Answers1