Unity + OpenXR + OVRSpatialAnchor + Passthrough = Wobbly models

Question

Previously we were using the Oculus XR Plugin with OVRSpatialAnchors and passthrough and all was working as intended. We needed to switch from the Oculus XR Plugin to OpenXR (for a separate package) and managed to get everything working except for when we put an OVRSpatialAnchor on a model with passthrough active, the model appears to be very wobbly compared to it's intended stable state. Once the spatial anchor has been removed off the model it appears much more stable but obviously doesn't have the other characteristics of having a spatial anchor that we need.

Does anyone know how to fix this? I saw this post with a similar issue and I updated the AndroidManifest.xml via the Oculus tools menu but that did not seem to fix it.

Potentially relevant details:

• Unity 2020.3.36
• MRTK v2.7.2
• OpenXR plugin v1.5.3
• Oculus XR plugin v1.12.0
• Oculus Integration v53.1
• Minimum Android API Level: Android 10 (API level 29)
• OpenXR settings:
  • Render mode: Single Pass Instanced/Multi-view
  • Depth Submission Mode: Depth 16 Bit
  • Interaction Profiles: Oculus Touch Controller Profile
  • OpenXR Feature Groups:
    • Hand Tracking
    • MetaXR Feature
    • Oculus Quest Support

I have tried turning off passthrough and saw that the models became stable again but that is not acceptable. I tried updating the AndroidManifest.xml as stated in my post but that did not change anything. My code hasn't changed going from the Oculus XR Plugin to the OpenXR Plugin but it can be summed up to just adding the OVRSpatialAnchor component:

model.AddComponent<OVRSpatialAnchor>();

EDIT: I built a minimal project using the same settings as above except I removed the MRTK and the Hand Tracking from the OpenXR Feature Groups. This time I didn't use the passthrough and still had the same issue, as soon as I remove the OVRManager from the scene then the wobblyness goes away (and the functionality of the spatial anchors I'm assuming). So there appears to be some conflict with OpenXR and the OVRSpatialAnchor's

Answer 1

I'm guessing that if you're switching to the OpenXR plugin, you also need to switch from OVRSpatialAnchor to an OpenXR equivalent. Presumably OVRSpatialAnchor will be getting it's pose information from the Oculus API, which won't be in perfect sync with the OpenXR API.

You can't mix and match OpenXR-based and Oculus-based plugins, because part of each API is fetching the expected pose for the next frame and then submitting the rendered frame with the pose information you rendered it at. Presumably Oculus plugins will be following one code path fetch these poses in Unity while OpenXR will be following another, and if you're using OpenXR to do the rendering then the Oculus poses will simply be wrong, or out of date by a tiny fraction, thus causing your wobble.

A quick search shows that there is an example of a Unity OpenXR SpatialAnchor application here so that might set you on the right course.

Update:

I have tried turning off passthrough and saw that the models became stable again but that is not acceptable

This is a tricky determination to make. The human eye is far more sensitive to objects moving against a background, than it is to determining the absolute motion of objects relative to head movement. My point is that the models may be rendering exactly the same, but without the passthrough imagery behind them to compare against you can't perceive the wobble any more.