Camera calibration at image corners

Question

I'm using OpenCV and working on calibrating a camera, testing a charuco-type calibrator for the first time.

I had many expectations about this calibrator, thinking with it I could get much closer to the corners of the image than with a standard chessboard, since charuco does not need to be seen entirely. This should in theory help the modeling of the projective system, as the optical deformations are greater in the corners.

The results I get leave me a little puzzled though. The interpolateCharucoCorners function, never returns me the last row/column of the chessboard, making it impossible to use those areas of the image for calibration.

Just to give you an idea I enclose the following image, where I highlighted the area described by the points extracted by interpolateCharucoCorners (the area is the convex hull of the whole set of points).

Charuco Board:

Charuco Board

As you can see the last column to the right of markers has not been extracted. Can you help me explain why? How do I include those pixels in the calibration? ChArUco corners are extracted as follows.

That's my code, but it's the same as in the example:

image_grayscale = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
corners, ids, rejected_points = cv2.aruco.detectMarkers(image_grayscale, board.dictionary)
_, charuco_corners, charuco_ids = cv2.aruco.interpolateCornersCharuco(corners, ids, image_grayscale, board)

If you don't find anyone who know about it, you can still look at the opencv code (it is open source) and maybe adapt it to your needs. — Micka, Jun 23 '20 at 07:21
According to doc, interpilateCharucoCorners is useful when you already have the calibration. Because with a pinhle camera, you can correctly interpolate the missing corners. Without pinhole camera, you can't guess the distortion correctly, so you can't interpolate the corners?!? — Micka, Jun 23 '20 at 07:22
according to these images, it looks like it is wanted that only the inner part of the chessboard is detected (similar to the general chessboard detection): https://docs.opencv.org/3.4/df/d4a/tutorial_charuco_detection.html — Micka, Jun 23 '20 at 07:25
Thank you Micka. I thought the outer part was automatically discarded, too. This could be because only X-shaped corners are accepted (90 degrees white + 90 degrees black + 90 degrees white + 90 degrees black). If you look at my image though, I purposely left a column of markers outside the FOV hoping to use the internal makers, but it didn't work. — Andrea Quattrini, Jun 23 '20 at 08:50
I just found out that the parameter minMarkers (number of adjacent markers that must be detected to return a charuco corner) does the job. https://docs.opencv.org/3.4/d9/d6a/group__aruco.html#gadcc5dc30c9ad33dcf839e84e8638dcd1. Setting it to zero, I solved my problem.. — Andrea Quattrini, Jun 23 '20 at 08:59
sorry, I didnt have a look at the right part of the image and thought you were meaning the missing outer-left corners already. Can you add an answer with the solution and an image? I will like to upvote it and maybe it will help me in the future. — Micka, Jun 23 '20 at 10:06

score 3 · Accepted Answer · answered Jun 23 '20 at 12:14

I finally found the solution. The markers near the edge can be extracted by setting the parameter minMarkers = 0 in interpolateCornersChaurco. From docs:

MinMarkers: number of adjacent markers that must be detected to return a charuco corner.

https://docs.opencv.org/3.4/d9/d6a/group__aruco.html#gadcc5dc30c9ad33dcf839e84e8638dcd1

This is the resulting image:

Camera calibration at image corners

1 Answers1