How to recognize WPF drawing visuals in UIAutomation?

Question

Our application has a canvas, to which we add the drawing visuals (like lines, polygons etc)

// sample code

var canvas = new Canvas(); // create canvas
var visuals = new VisualCollection(canvas); // link the canvas to the visual collection
visuals.Add(new DrawingVisual()); // add the visuals to the canvas
visuals.Add(new DrawingVisual());

Our goal is to add these visuals to the canvas via automation and validate that they are properly added. We use a framework that is based on Microsoft's UIAutomation.

When using a tool like "Inspect" to inspect the visual structure, I couldnt locate the canvas. Did some research and figured out that you need to override the OnCreateAutomationPeer method from UIElement, and return applicable AutomationPeer object to be able to be able to see that in automation.

Made the change and now I can see the canvas, however I cant still see any of the visuals added under the canvas.

Can anyone help me understand what the issue is?

Things tried / alternatives:

1. Tried to employ the OnCreateAutomationPeer technique, but the DrawingVisuals dont derive from UIElement, and I cant add UIElements to Canvas.VisualCollection.
2. Image recognition is an option, but we are trying to avoid it for performance/maintenance considerations.

Answer 1

Only UIElement can be seen from UI Automation (like you have seen, OnCreateAutomationPeer starts from this class, not from the Visual class).

So you need to add UIElement (or derived like FrameworkElement) to the canvas, if you want it to be usable by UIAutomation.

You can create your own class like described here: Using DrawingVisual Objects or with a custom UserControl or use an existing one that suits your need but it must derive from UIElement somehow.

Once you have a good class, you can use the default AutomationPeer or override the method and adapt more closely.

If you want to keep Visual objects, one solution is to modify the containing object (but it still needs to derive from UIElement). For example, here if I follow the article in the link, I can write a custom containing object (instead of a canvas of your sample code so you may have to adapt slightly) like this:

public class MyVisualHost  : UIElement
{
    public MyVisualHost()
    {
        Children = new VisualCollection(this);
    }

    public VisualCollection Children { get; private set; }


    public void AddChild(Visual visual)
    {
        Children.Add(visual);
    }

    protected override int VisualChildrenCount
    {
        get { return Children.Count; }
    }

    protected override Visual GetVisualChild(int index)
    {
        return Children[index];
    }

    protected override AutomationPeer OnCreateAutomationPeer()
    {
        return new MyVisualHostPeer(this);
    }

    // create a custom AutomationPeer for the container
    private class MyVisualHostPeer : UIElementAutomationPeer
    {
        public MyVisualHostPeer(MyVisualHost owner)
            : base(owner)
        {
        }

        public new MyVisualHost Owner
        {
            get
            {
                return (MyVisualHost)base.Owner;
            }
        }

        // a listening client (like UISpy is requesting a list of children)
        protected override List<AutomationPeer> GetChildrenCore()
        {
            List<AutomationPeer> list = new List<AutomationPeer>();
            foreach (Visual visual in Owner.Children)
            {
                list.Add(new MyVisualPeer(visual));
            }
            return list;
        }
    }

    // create a custom AutomationPeer for the visuals
    private class MyVisualPeer : AutomationPeer
    {
        public MyVisualPeer(Visual visual)
        {
        }

        // here you'll need to implement the abstrat class the way you want
    }
}