Is this a normal amount of instability for an ARUCO board estimation? (video example)

Question

I'm trying to use ARUCO boards to assist with indoor robot navigation and I'm having trouble with the pose estimation Z axis flickering when the board is most head-on to the camera. Please see the issue here - watch how the blue axis flickers back and forth when facing the camera, but is more stable at other angles:

https://drive.google.com/file/d/1l8uqKLbA9R4ij_kjPcYeGPUObR_sHkGn/view?usp=sharing

In my navigation paradigm, I assumed the best position-lock on the board would be from dead in front, but that appears to be the worst under my current setup.

• This video is with sub-pixel corner refinement turned on.
• The individual markers are being pose estimated separately from the board, so we can see the issue appears on the individual markers and not just with the board itself.
• The blue axis instability is always horizontal, even if I turn the marker 90 degrees, the jitter still happens accross the horizontal - no such jitter occurs when tilting across 0 going up and down, only left and right.
• I'm passing the rvec and tvec straight from the detection into opencv to draw the markers, like this:

Is this just... how aruco codes/boards look head on? Or is it a problem of my board design, or something else? Would love to hear from someone who's got some experience with jitter (or stability) of their own, thanks!

Answer 1

It's more of a CV domain specific problem rather than coding problem. Nevertheless I'd answer this question for your interest.

To make the result more precise, you need to enhance the system in the following ways:

1. Camera calibration

I think your camera has a small focal length and thus has some distortion. The algorithm use shape distortion to calculate the posture of marker, and thus distortion of your camera will affact the precision by a great deal.

2. Rigid marker

The marker is printed on an A4 paper which is very soft. For the same reason this will affact the precision of posture inferring. Print the marker on a cardboard, or stick the printed marker on a hardboard, or you can place the marker on the desk and move the camera for testing.

3. Higher resolution and better lighting conditions

I'm not sure if the video is compressed when uploading but apparently higher resolution leads to higher precision. The uploaded video is of poor lighting and resolution.

Your computer and the algorithm seems to be working well because the framerate is pretty well. I think the main problem is your experiment setup.

The blue axis instability is always horizontal, even if I turn the marker 90 degrees, the jitter still happens accross the horizontal - no such jitter occurs when tilting across 0 going up and down, only left and right.

The jitter comes from subtle distorion of your marker, and when the marker is perpendicular to your camera main axis, the projection of marker is closer to square shape, the derivative of 'blue axis' over shape distortion is larger when marker is '90 degree'. Thus it's more sensitive to those distortions. That's why the blue axis jitters more seriously when it comes to 90 degree position.

Here are some successful tests available. one and another

Answer 2

The problem is most likely ought to bad camera parameters. Have you calibrated it and checked its quality?. Subpixel stability and having a high quality target will not help you much if the calibration is not accurate.

Just for the sake of completeness (I know you have been through them) OpenCV describes in its documentation and tutorial how to checkproof your calibration visually and numerically

https://docs.opencv.org/3.4/d4/d94/tutorial_camera_calibration.html https://docs.opencv.org/master/dc/dbb/tutorial_py_calibration.html