How do I convert from mouse coordinates to pixel coordinates of a TransformedBitmap?

Question

My question is similar to How to get correct position of pixel from mouse coordinates?, with the added caveat that the image is potentially a TransformedBitmap, where flips and rotations could be applied, and still return the pixel coordinates of the original image.

My Window's design looks like this:

    <DockPanel>
        <Label DockPanel.Dock="Bottom" Name="TheLabel" />
        <Image DockPanel.Dock="Top" Name="TheImage" Stretch="Uniform" RenderOptions.BitmapScalingMode="NearestNeighbor" MouseMove="TheImage_MouseMove" />
    </DockPanel>

and the codebehind looks like this:

        public MainWindow()
        {
            InitializeComponent();

            const int WIDTH = 4;
            const int HEIGHT = 3;
            byte[] pixels = new byte[WIDTH * HEIGHT * 3];
            pixels[0] = Colors.Red.B;
            pixels[1] = Colors.Red.G;
            pixels[2] = Colors.Red.R;
            pixels[(WIDTH * (HEIGHT - 1) + (WIDTH - 1)) * 3 + 0] = Colors.Blue.B;
            pixels[(WIDTH * (HEIGHT - 1) + (WIDTH - 1)) * 3 + 1] = Colors.Blue.G;
            pixels[(WIDTH * (HEIGHT - 1) + (WIDTH - 1)) * 3 + 2] = Colors.Blue.R;
            BitmapSource bs = BitmapSource.Create(WIDTH, HEIGHT, 96.0, 96.0, PixelFormats.Bgr24, null, pixels, WIDTH * 3);
            TheImage.Source = bs;
        }

        private void TheImage_MouseMove(object sender, MouseEventArgs e)
        {
            Point p = e.GetPosition(TheImage);
            if (TheImage.Source is TransformedBitmap tb)
                TheLabel.Content = tb.Transform.Inverse.Transform(new Point(p.X * tb.PixelWidth / TheImage.ActualWidth, p.Y * tb.PixelHeight / TheImage.ActualHeight)).ToString();
            else if (TheImage.Source is BitmapSource bs)
                TheLabel.Content = new Point(p.X * bs.PixelWidth / TheImage.ActualWidth, p.Y * bs.PixelHeight / TheImage.ActualHeight).ToString();
        }

When hovering in the bottom-right corner (which I colored blue for easy tracking) of the non-transformed image, you correctly see the coordinates of (~4, ~3), which are the image dimensions.

However, once you apply a transform, for example changing TheImage.Source = bs; to TheImage.Source = new TransformedBitmap(bs, new RotateTransform(90.0));, hovering over the blue instead gives (~4, ~0).

I think one could look at the actual matrix values of the transform, and determine how to adjust the point in all the various cases, but it seems like there should be an easier solution using the inverse transform.

Clemens · Accepted Answer · 2021-11-23T10:54:45.857

When an Image element renders a TransformedBitmap, the center point of the Transform is effectively ignored and the bitmap is shifted into an appropriate coordinate range.

Since this shift is not applied to the transfomation of a Point, you have to compensate it, e.g. like this:

var p = e.GetPosition(TheImage);

if (TheImage.Source is BitmapSource bs)
{
    p = new Point(
        p.X * bs.PixelWidth / TheImage.ActualWidth,
        p.Y * bs.PixelHeight / TheImage.ActualHeight);

    if (TheImage.Source is TransformedBitmap tb)
    {
        var w = tb.Source.PixelWidth;
        var h = tb.Source.PixelHeight;

        p = tb.Transform.Inverse.Transform(p);

        p = new Point((p.X + w) % w, (p.Y + h) % h);
    }

    TheLabel.Content = $"x: {(int)p.X}, y: {(int)p.Y}";
}

The above code will however not work correctly when the Transform center is set to a value other than (0,0).

In order to make it work for Transforms with arbitrary center point, you have to clear the offset values in the transform matrix:

var p = e.GetPosition(TheImage);

if (TheImage.Source is BitmapSource bs)
{
    p = new Point(
        p.X * bs.PixelWidth / TheImage.ActualWidth,
        p.Y * bs.PixelHeight / TheImage.ActualHeight);

    if (TheImage.Source is TransformedBitmap tb)
    {
        var w = tb.Source.PixelWidth;
        var h = tb.Source.PixelHeight;
        var t = tb.Transform.Value;
        t.Invert();
        t.OffsetX = 0d;
        t.OffsetY = 0d;

        p = t.Transform(p);

        p = new Point((p.X + w) % w, (p.Y + h) % h);
    }

    TheLabel.Content = $"x: {(int)p.X}, y: {(int)p.Y}";
}

Thanks! If one embeds the `Image` in a `Grid` which has a background and moves the `MouseMove` event to the `Grid`, the pixel coordinates are no longer accurate when outside the bounds of the image since it only gives answers from 0..width and 0..height. Any idea how to fix this? — Craig W, Nov 23 '21 at 15:56
There are multiple ways. Most simple, attach the event handler to another nested parent element that has the same dimensions as the Image. Otherwise there are methods like TransformToDescendant or similar. — Clemens, Nov 23 '21 at 16:03
Not sure I follow. I can translate the point from the parent grid to the image no problem. But for example when the user hovers to the right of the rotated image, I'd like the y-coordinate to be negative to reflect that they are above the bounds of the image in the original orientation. Should I edit my question or start a new one? — Craig W, Nov 23 '21 at 16:53

How do I convert from mouse coordinates to pixel coordinates of a TransformedBitmap?

1 Answers1

Linked