2

I need to be able to get the coordinates of a pixel under the mouse to ensure that when I calculate the distance between 2 points, it ends up being the correct distance.

My issue is that when I try to translate the texture, the mouse (drag based translation) never appears to "grab" a specific pixel.

What I am trying to achieve:

I am trying to create an image viewer program using OpenGL 4. I need to get the basic pan and zoom functionality. I am trying to ensure that when I am panning, the following is always true:

  • The pixel that is under the mouse when the click is initiated (mouse down) is the same one under the mouse when the click is released (mouse up).
  • The pixel that is under the mouse when the user uses the scroll wheel to zoom is always the center point of the zoom (scale the image and then translate to ensure that pixel is in the center of the viewable window)

What I have tried:

I have tried taking the screen coordinates for the start and endpoints and dividing them by the screen width/height for x and Y to get the "delta" or change. This never made it so the mouse would appear to anchor to the image.

I have tried then multiplying that number by the zoom level (value between 0 and 1). That seemed to help a bit, but not too much.

I also tried (most current attempt) to use the image height/width instead of the screen height/width

The last thing I have been trying to work with is using GL.ReadPixels, but it needs an IntPtr as the final parameter. I have not been able to find a good example of how that code would work in C#. There are a bunch of C++ examples, but almost no C# examples.

DrawBuffer code:

This is the code that calls all of the OpenGL code. The information that I need is how to ensure that the correct values get put into translationVector.

Extra information

The image viewer is built to handle two images (that are to be used for stereoscopic viewing). The main "image" that is going to be viewed is the left image.

The corrected offset is code that predates me, so I am not 100% sure about the following:

  • correctedOffset is the delta of a mouse movement in addition to an adjustment if there are 2 images.
  • normalizedOffset is a delta between 0 and 1 to be used to accurately translate the MVP matrix

The following classes are based on OpenGL tutorials by TheCherno:

  • VertexBuffer
  • VertexBufferLayout
  • IndexBuffer
  • Shader
    • This is the type of _shaderProgram
  • Renderer
  • Texture
    • This is the type of _imageTexture
  • VertexArray
GL.Viewport(new Size(OrthoBounds.Width, OrthoBounds.Height));

Vector3 translationVector = new Vector3();

// The model matrix controls the position of the model (object in your frame)
Matrix4 model;

// The view matrix controls the position of the "camera". In our use, we do not need to set this.
Matrix4 view = new Matrix4();

// The projection matrix is the field of view.
Matrix4 projection = Matrix4.CreateOrthographic(OrthoBounds.Left, OrthoBounds.Top, -1, 1);

// The identity matrix gives a starting point for matrix calculations.
Matrix4 identity = Matrix4.Identity;

//TODO: UI CHANGE
GL.ClearColor(0.09f, 0.09f, 0.09f, 1.0f);
GL.Clear(ClearBufferMask.ColorBufferBit);

// todo: if left and right image are different sizes?
// fit image to inside of control
float scale = Math.Min(
    (float)Bounds.Width / LeftImage.Image.ImageSize.Width,
    (float)Bounds.Height / LeftImage.Image.ImageSize.Height);
Size imageSize = new Size(
    (int)(LeftImage.Image.ImageSize.Width * scale),
    (int)(LeftImage.Image.ImageSize.Height * scale));

// use scaled image to caclulate bounds
Rectangle imageBounds = new Rectangle(Point.Empty, imageSize);
imageBounds.Offset(-imageSize.Width / 2, -imageSize.Height / 2);

// todo: maybe use .Offset here, but it would need property changes
PointF correctedOffset = new PointF(
    OrthoTranslate.X + (RegisterTranslate.X * (eyeSide == LateralityType.Left ? -1 : 1)),
    OrthoTranslate.Y + (RegisterTranslate.Y * (eyeSide == LateralityType.Right ? -1 : 1))
);

/*
 *  List of vertices that we will need to render the image
 *  Layout: 
 *      Vertex X Coord, Vertex Y Coord, Texture X Coord, Texture Y Coord
 */
float[] vertices = new float[]
{
    imageBounds.Left, imageBounds.Top, 1.0f, 0.0f, // 0
    imageBounds.Right, imageBounds.Top, 0.0f, 0.0f, // 1
    imageBounds.Right, imageBounds.Bottom, 0.0f, 1.0f, // 2
    imageBounds.Left, imageBounds.Bottom, 1.0f, 1.0f // 3
};

/*
 * List of indices that we will use to identify which vertices 
 * to use for each triangle. This reduces the number of times we
 * would need to specify vertices as there are no duplicates.
 */
int[] indices =
{
    1, 2, 3,
    3, 0, 1,
};

var normalizedOffset = new PointF(correctedOffset.X / imageBounds.Width,
    correctedOffset.Y / imageBounds.Height);

translationVector = new Vector3(normalizedOffset.X, normalizedOffset.Y, 0.0f);

// Translate the model matrix
model = Matrix4.CreateTranslation(translationVector);

// Scale the model matrix
model *= Matrix4.CreateScale(Zoom, Zoom, 1);

/*
 * The model, view, projection matrix is the final matrix that contains 
 * the information from all three matrices to form the one that will be 
 * used to render the object on the screen.
 */
_mvpMatrix = projection * identity * model;

Vector3.Unproject(translationVector, 0.0f, 0.1f, 1.0f, 1.0f, 0.0f, 1.0f, _mvpMatrix);

_shaderProgram.SetUniformMat4f("u_MVP", _mvpMatrix);

_shaderProgram.SetUniform1("brightness", Brightness - 1f);
_shaderProgram.SetUniform1("contrast", Contrast);

// set color channel parameter using glsl
_shaderProgram.SetUniform4f("channels", new Vector4(_channelState[RasterColorChannel.Red], _channelState[RasterColorChannel.Green], _channelState[RasterColorChannel.Blue], 1.0f));

// set texture unit identifiers
_shaderProgram.SetUniform1("image", _imageTexture.Id);
_shaderProgram.SetUniform1("overlay", _imageTexture.Id);

// update overlay parameter for shader
_shaderProgram.SetUniform1("drawOverlay", _drawOverlays ? 1 : 0);

_renderer.Clear();

//The vertex array
VertexArray va = new VertexArray();

//The vertex buffer
VertexBuffer vb = new VertexBuffer(vertices, 4 * 4 * sizeof(float));

//The vertex buffer layout
VertexBufferLayout layout = new VertexBufferLayout();

//The index buffer
IndexBuffer ib = new IndexBuffer(indices);

//Add a grouping of two for the vertex XY coordinates
layout.AddFloat(2);

//Add a grouping of two for the texture XY coordinates
layout.AddFloat(2);

//Add the vertex buffer with the associated layout to the vertex array object
va.AddBuffer(vb, layout);


_imageTexture.Bind();

_renderer.Draw(va, ib, _shaderProgram);

_imageTexture.Unbind();

Outcomes

What I Expected to happen vs what did happen

Scenario:

At the normal 0 zoom level, I dragged from the left side to the right side (to drag the image right).

  • Expected outcome: ** The mouse would stay anchored to a specific part of the image as it was moving.
  • Actual outcome: ** The mouse would stay close and get further away from the original point as I dragged further to the right.

Other things I noticed

  • The more I zoom in, the more accurate it appears
    • This means that it stays closer to the original point that I dragged from
  • It will still move the image even if I drag from outside of the image bounds
    • I want it only to move the image when you are dragging from within the image.

Let me know if there is any other information that may be helpful!

DaDuStMaN20
  • 137
  • 2
  • 11

0 Answers0