0

I'm experiencing a strange behavior in function cv2.interpolateCornersCharuco especially when the board is partially occluded, see the following picture:

enter image description here

Charuco board is an 8x11 board using 5x5 Aruco as dictionary. Somehow, if I run this code:

    board = cv2.aruco_CharucoBoard.create(8, 11, 0.075, 0.058, cv2.aruco.getPredefinedDictionary(cv2.aruco.DICT_5X5_250))

    detection_params = cv2.aruco.DetectorParameters_create()
    detection_params.cornerRefinementMethod = cv2.aruco.CORNER_REFINE_CONTOUR

    image_grayscale = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)
    corners, ids, rejected_points = cv2.aruco.detectMarkers(image_grayscale, board.dictionary, parameters=detection_params)

    _, charuco_corners, charuco_ids = cv2.aruco.interpolateCornersCharuco(corners, ids, image_grayscale, board,
                                               cameraMatrix=None, distCoeffs=None, minMarkers=1)

    debug_image = cv2.aruco.drawDetectedMarkers(image, corners, ids, borderColor=(0, 255, 0))
    debug_image = cv2.aruco.drawDetectedCornersCharuco(debug_image, charuco_corners, charuco_ids, (0, 0, 255))

    fig = plt.figure()
    plt.imshow(debug_image)
    plt.pause(0.5)
    plt.close()

I got this result, which is clearly wrong and lead to a bad pose estimation. This effect is reduced if all the markers are visible.

enter image description here

Andrea Quattrini
  • 123
  • 9

1 Answers1

1

The problem is how the calibration tables are generated - OpenCV made a change so that charuco boards always have a black box in the upper left corner.

Related issue: https://github.com/opencv/opencv/issues/23152

As a result, legacy boards (generated in OpenCV version 4.5.5 or earlier) will not be correctly detected. In the current version (4.7.0) there is no way to fix this bug. It is worth noting that the problem only affects arrays with an even number of rows.

You check generated board appearance using the following code.

import cv2

ARUCO_DICTIONARY = cv2.aruco.getPredefinedDictionary(cv2.aruco.DICT_5X5_250)
board = cv2.aruco.CharucoBoard((11, 8), 0.075, 0.058, ARUCO_DICTIONARY)
image = board.generateImage((640, 400))

cv2.namedWindow("Charuco board")
cv2.imshow("Charuco board", image)
cv2.waitKey(0)

Charuco board generated with OpenCV 4.7.0

In next versions of OpenCV, the method setLegacyPattern will be added to the CharucoBoard class, which allows to specify that the table was generated in a old-fashioned way.

CharucoBoard documentation: https://docs.opencv.org/4.x/d0/d3c/classcv_1_1aruco_1_1CharucoBoard.html

Juliusz Tarnowski
  • 23
  • 4