How to detect if InputSystem.EnhancedTouch.Finger is over UI Toolkit Element?

Question

I'm using InputSystem.EnhancedTouch for my game and I have a few buttons that are available through UI Elements. When I click on the UI Elements buttons it activates the InputSystem thus creating a bad experience. I want to filter out all the InputSystem.EnhancedTouch events that come through UI.

TL;DR - I want UI Elements to block InputSystem.EnhancedTouch events from triggering

I have found quite some resources but nothing really works. Unity and some other people say to use EventSystem.current.IsPointerOverGameObject but it doesn't work and throws the following warning(I think this is meant to work with normal input only, not enhanced one)

I have tried a few other solutions such as UIDocument.rootVisualElement.panel.Pick or EventSystem.current.RaycastAll but nothing seems to work, or return any consistent data that can be used.

InputSystem.EnhancedTouch binding

    private void Awake()
    {
        EnhancedTouchSupport.Enable();
        Touch.onFingerDown += OnFingerDown;
        Touch.onFingerMove += OnFingerMove;
        Touch.onFingerUp += OnFingerUp;
    }