-1

I have correct correspondances between consecutive frames, and need to estimate the transformation between them to generate a trajectory. The following C++ pipeline, the generated trajectory goes no sense.

auto EssentialMatrix = cv::findEssentialMat(points_previous,
                                            points_current, 
                                            camera_focal_length, 
                                            camera_principal_point,
                                            cv::RANSAC, 
                                            0.999, 
                                            1.0, 
                                            mask);

auto inliers = cv::recoverPose(EssentialMatrix, 
                points_previous, 
                points_current, 
                CameraMatrix, 
                R, 
                t, 
                mask);

t_pos_ = t_pos_ + 1.0 *(R_pos_*t);  
R_pos_ = R * R_pos_;

So, my question is: how to correctly recover the transformation between two consecutive frames with C++ OpenCV utilities? Are additional steps needed to do so?

Employee
  • 3,109
  • 5
  • 31
  • 50
  • can you show your working python code? What is R_pos_ and t_pos_? How are they initialized? – Micka Jun 25 '20 at 16:04
  • so R and R_pos_ as well as t and t_pos_ are equal after the first call? Can you try to isolate the same functional parts from your python code? Didn't you use opencv in python for that task? – Micka Jun 25 '20 at 16:27
  • did you undistort the points? https://stackoverflow.com/questions/31290414/undistortpoints-findessentialmat-recoverpose-what-is-the-relation-between-the – Micka Jun 25 '20 at 18:35
  • if python and C++ pipelines are doing the same, can you try to compare intermediate results? Probably, either findEssentialMat or recoverPose are giving different results to your python functionality, or the inputs differ. If you can isolate the poblem, try to find differences in the implementation (maybe parameters have to be used differently? Maybe opencv optimizations have reduced quality?) – Micka Jun 26 '20 at 14:02

1 Answers1

0

I really have no idea whats is your problem. Try to give like whats the input, whats "your expected output" whats the actual output.

But i do know where to find sample https://github.com/avisingh599/mono-vo

try to take a look at sample here. It is pure 2D to 2D opencv based estimation. only thing different is that they use dataset saclaing to adjust the translational vector scale. 2D to 2D alone only give a relative translation.

http://www.youtube.com/watch?v=homos4vd_Zs

enter image description here

Dr Yuan Shenghai
  • 1,849
  • 1
  • 6
  • 19
  • what is the thing comming out???? you keep saying whats the correct way or make no sense. whats the data coming out that makes you think make no sense? det(R) = 1? if so that's make no sense. if not. you are not express your problem clearly – Dr Yuan Shenghai Jun 26 '20 at 12:57
  • yes go on give example. jumps means what? R value sudden give 90 degree yaw or someother thing? by right R should be the same. t value might changes from time to time – Dr Yuan Shenghai Jun 26 '20 at 13:05
  • Hello @dr yuan. I have a similar issue as OP, but not exactly. For ```recoverPose```, if I pass my camera intrinsics as arguments, I notice that the number of points returned (that pass the cheirality check) is zero or close to zero. But when I do not pass the camera intrinsics, the number of points returned are a lot. I do not understand it at all, could you help me out? – OlorinIstari Jun 27 '20 at 10:17
  • @ShrutheeshRamanIyer can you open an new question. Add on your python code. you raw input 2 images. and your camera matrix? use cout function to print out. there are too many possibility, without looking at details, no way i can make guess – Dr Yuan Shenghai Jun 27 '20 at 18:39
  • Hey @dr yuan. I had opened a new question, but I got no response. I have raised an issue on github as well. https://github.com/opencv/opencv/issues/17690 Please have a look if possible. Thanks! – OlorinIstari Jun 30 '20 at 04:32
  • saw the image. i got a feeling its the tracking error. which is common in visual odoemtry study. I`ll run those code after I'm back to lab – Dr Yuan Shenghai Jun 30 '20 at 05:07
  • Hi @dr yuan. Thanks a ton! – OlorinIstari Jun 30 '20 at 05:55