I'm quite the beginner using Unity. For a project, I placed ArUco markers on cubes, placed in front of a camera. In Unity, I recreated the setup, with the location of the Cubes and the ARCamera being the same as in real life, the camera being the origin. Using the OpenCVforUnity Package, I used it's MarkerBased AR Example (https://assetstore.unity.com/packages/templates/tutorials/markerbased-ar-example-29678) to attach the respective ArUco markers to the virtual cubes in the scene. In my expecation, since the distances in the scene are the same as in real life, the cubes should stay in exactly the same position, unless they are moved in real life, in which case the virtual cubes move the same.
Now my expectations were not met. While the virtual cubes move if the real cubes move, the markers are detected at entirely different locations as where the cubes are currently, so the virtual cubes are at "wrong" locations in the scene. This is not a small error, the results are about 17 meters off.
It seems to me, like there is some conversion from one coordinate system to the other that I'm missing here, but I'm not sure which one. I already read that Unity uses a left handed coordinate system, so the necessary inversions for that are all applied. I hope the problem is understandable, if you need any more information I will try to provide anything necessary.
