I am trying to find the positions of multiple monitors in 3d space. For that purpose I display charuco boards and take a short video of the whole setup. Sample frame here (cropped down from 4k to preserve privacy):
Code is taken verbatim from the OpenCV charuco tutorials:
- https://docs.opencv.org/master/da/d13/tutorial_aruco_calibration.html
- https://docs.opencv.org/3.4/df/d4a/tutorial_charuco_detection.html
The only change I made was that I created all six boards from separate parts of a big aruco dictionary so that every single marker is unique.
Calibration reports an error of 3-4 (which I believe is in pixels, so on a 4k video isn't too bad).
90% of time this works fine, but in some frames the detection fails spectacularly for no apparent reason as is visible in the lower right screen in the image above. Note that the actual Aruco markers (red corners, green frames, blue ids) get detected well, but the program fails to detect the pose of the charuco board (green markers, rgb axis). This happens sporadically for one or two frames, but I would prefer it not to happen at all.
Any ideas?
