0

In OpenCV, I am using a Charuco board, have calibrated the camera, and use estimate to get rvec and tvec. (similar to the sample code). I am using a stationary board, with a camera moves around. I'm a noob with this, so please bear with me if this is something simple I am missing.

I understand that I can use Rodrigues() to get the 3x3 rotation matrix for the board orientation from rvec, and that I can transform the tvec value into world coordinates using worldPos = -R.t() * tvec (in python) to get position of camera

It seems that I got everything i need, however, if I rotate the camera physically, WorldPos also changes hugely! I am sure it’s not because the camera has been moved since I tried many times.

I don’t know where the problem is, thanks for your suggestion.

And the code is below(in python).

self.retval, self.rvec, self.tvec =   aruco.estimatePoseBoard(self.corners, self.ids, board, self.cameraMatrix, self.distanceCoefficients)
self.dst, jacobian = cv2.Rodrigues(self.rvec)
self.rvec_trs = self.dst.transpose()
self.worldPos = - self.rvec_trs * self.tvec
self.worldPos = [self.worldPos[0][0],self.worldPos[1][1], self.worldPos[2][2]]
linfeng chen
  • 522
  • 6
  • 5
  • Try to ask here: http://answers.opencv.org/questions/ – Ugnius Malūkas Oct 16 '18 at 10:31
  • what do you mean with "if I rotate the camera"? physically? how much is hugely? you should explain better with maybe an example and numbers and the code that you have used, for people to understand the problem better. – api55 Oct 16 '18 at 10:53
  • are you sure that rvec and tvec are the camera extrinsics and not the object/board pose relative to the camera? – Micka Oct 16 '18 at 11:21
  • @ api55 yes, I rotate the camera physically. – linfeng chen Oct 17 '18 at 01:08
  • @Micka I think it's the board pose relative to the camera. I found that I need to get the camera extrinsic to estimate the worldPos of camera. So can you give me some suggestions? thank you! – linfeng chen Oct 17 '18 at 01:12
  • if you compose the 4x4 transformation from rvec and tvec you can just invert it to get the camera pose, afaik. – Micka Oct 17 '18 at 04:38
  • @Micka Thank you very much! I've solved that problem and can get world position of camera! – linfeng chen Oct 17 '18 at 06:34

1 Answers1

1

Finally I solved it with the help of @Micka. The code is below(in python), hope it can help others who have the same problem.

self.retval, self.rvec, self.tvec = aruco.estimatePoseBoard(self.corners, self.ids, board, self.cameraMatrix, self.distanceCoefficients
self.dst, jacobian = cv2.Rodrigues(self.rvec)
self.extristics = np.matrix([[self.dst[0][0],self.dst[0][1],self.dst[0][2],self.tvec[0][0]],
                             [self.dst[1][0],self.dst[1][1],self.dst[1][2],self.tvec[1][0]],
                             [self.dst[2][0],self.dst[2][1],self.dst[2][2],self.tvec[2][0]],
                             [0.0, 0.0, 0.0, 1.0]
                ])
self.extristics_I = self.extristics.I  # inverse matrix
self.worldPos = [self.extristics_I[0,3],self.extristics_I[1,3],self.extristics_I[2,3]]

But I still don't know why worldPos = -R.t() * tvec cannot get the right answer. Hope someone can tell me. Thanks!

linfeng chen
  • 522
  • 6
  • 5
  • it doesnt matter whether the camera is stationary and the objects move or the objects are stationary and the camera moves, the imagea will be the same. BUT the transformations applied to the objects or the camera are different (inverse) – Micka Oct 17 '18 at 06:56
  • `worldPos = -R.t() * tvec` doesn't work because the `*` operator just does an element-by-element multiplication instead of a real matrix multiplication. You need to use the `@` operator instead. – Josh Davis Sep 30 '21 at 18:13