0

I am working on a project to detect the 3D location of the object. I have two cameras set up at two corners of the room and I have obtained the Fundamental matrix between them. These cameras are internally calibrated. My images are 2592 X 1944

K = [1228 0 3267 0 1221 538 0 0 1 ]

F = [-1.098e-7 3.50715e-7 -0.000313 2.312e-7 2.72256e-7 4.629e-5 0.000234 -0.00129250 1 ]

Now, How do I proceed so that given a 3D point in space, I should be able to get points on the image which correspond to the same object in the room. If I can obtain the right projection matrices (with correct scale) I can use them later as inputs to OpenCV's traingulatePoints function to obtain the location of the object.

I have been stuck at this since a long time. So, please help me.

Thanks.

amoghesturi
  • 238
  • 3
  • 13
  • If your camera calibration matrix is written rowise, i.e., `K = [1228 0 3267; 0 1221 538; 0 0 1]` in Matlab notation, I think it is very awkward. Your image size is `2592 x 1944` and the principal point of your camera calibration which is usually the near the midpoint of the image is at position `(3267, 538)`. I think your camera calibration is incorrect... – who9vy Oct 09 '13 at 21:37

3 Answers3

2

From what I gather, you have obtained the Fundamental matrix through some means of calibration? Either way, with the fundamental matrix (or the calibration rig itself) you can obtain the pose difference via decomposition of the Essential matrix. Once you have that, you can use matched feature points (using a feature extractor and descriptor like SURF, BRISK, ...) to identify which points in one image belong to the same object point as another feature point in the other image. With that information, you should be able to triangulate away.

Hitesh Vaghani
  • 1,342
  • 12
  • 31
  • So, from the Fundamental Matrix, I calculated the essential matrix using E=K2'*F*K1. Then, my SVD needs a diagonal matrix such as (s,s,1) but my diagonal matrix is diag(1.3, 1.05, 0). I assume it is good enough and then find translation using T = last column of U and R could be U*D*V' or U*D'*V'. If i find out which of these combinations is the right one, I will have P1 = [I|0] and P2 = [R|T]. So, I read the scale is an ambiguity which can never be determined. I need to know the scale since I am trying to estimate the exact 3d co ordinates of the objects. Am i in the right direction? – amoghesturi Aug 12 '13 at 15:38
  • You can enforce this constraint that the diagonal matrix of the SVD is `diag(1,1,0)`, i.e., if the SVD of your essential matrix is `E=U*D*V'`, you can set the correct essential matrix `E'` to `E'=U*diag(1,1,0)*V'` as given in http://en.wikipedia.org/wiki/Eight-point_algorithm step 3. – who9vy Oct 09 '13 at 21:39
0

May be it will be simplier use cv::reprojectImageTo3D function? It will give you 3D coordinates.

Andrey Smorodov
  • 10,649
  • 2
  • 35
  • 42
  • My main concern was to obtain the projection matrices P1 and P2. I see that with the help of fundamental matrix F, I can only obtain canonical projection matrices but these are not up to real scale. Can you kindly guide me how I could obtain P1 and P2 which are good for real scale. – amoghesturi Aug 11 '13 at 16:16
  • Here is lecture about 3D reconstruction ( http://www.umiacs.umd.edu/~ramani/cmsc828d/lecture28.pdf ), and there is chapter "Reconstruction Ambiguities" that tells us that we can't compute scale from 2 views. – Andrey Smorodov Aug 11 '13 at 17:12
0

Sorry its not coming in size of comment.. so @user2167617 reply to your comment. Pretty much. A few pointers, though: the singular values should be (s,s,0), so (1.3, 1.05, 0) is a pretty good guess. About the R: Technically, this is right, however, ignoring signs. It might very well be that you get a rotation matrix which does not satisfy the constraint deteminant(R) = 1 but is instead -1. You might want to multiply it with -1 in that case. Generally, if you run into problems with this approach, try to determine the Essential Matrix using the 5 point algorithm (implemented into the very newest version of OpenCV, you will have to build it yourself). The scale is indeed impossible to obtain with these informations. However, it's all to scale. If you define for example the distance between the cameras being 1 unit, then everything will be measured in that unit.

Hitesh Vaghani
  • 1,342
  • 12
  • 31
  • Thanks for the reply. I found the rotation and translation matrix assuming that z is pointing from the camera center towards the scene. Then, I used triangulatepoints on matching points. When I find the distance between two points in the real scene and when I compare it with the distance between the same points in the triangulated image, it was possible to find the scale. There is a bit of error in the distance I obtain in this way. I assume it is because of the simple linear triangulation I have done without image rectification. I hope i will get good results with better triangulation. – amoghesturi Aug 16 '13 at 13:31
  • If its done than close the question by accepting one of the answer. – Hitesh Vaghani Aug 17 '13 at 05:13