21

I managed to acquire camera's intrinsic and extrinsic parameters using OpenCV, thus I have fx, fy, cx and cy. And I also have the screen / image's width and height.

But how do I create an OpenGL perspective projection matrix from these parameters?

glFrustrum shows how to create projection matrix, given Z near, Z far and the image width and height. But how do I include focal points and camera centers in this matrix?

enter image description here

sub_o
  • 2,642
  • 5
  • 28
  • 41

5 Answers5

13

You can check the following links

Kyle Simek's explanation

My explanation

Community
  • 1
  • 1
koshy george
  • 671
  • 6
  • 24
10

Here is the code to obtain the OpenGL projection matrix equivalent to a computer vision camera with camera matrix K=[fx, s, cx; 0, fy, cy; 0, 0, 1] and image size [W, H]:

glMatrixMode(GL_PROJECTION); // Select The Projection Matrix
glLoadIdentity();            // Reset The Projection Matrix
GLdouble perspMatrix[16]={2*fx/W,0,0,0,2*s/W,2*fy/H,0,0,2*(cx/W)-1,2*(cy/H)-1,(zmax+zmin)/(zmax-zmin),1,0,0,2*zmax*zmin/(zmin-zmax),0};
glMultMatrixd(perspMatrix);

NB: zmin and zmax represent the near and far Z clipping planes. This formulation assumes that the OpenGL world coordinate frame is chosen as follows:

enter image description here

The OpenGL camera is assumed to be located at the origin, looking towards positive Z axis, with a down vector collinear and towards the positive Y axis.

BConic
  • 8,750
  • 2
  • 29
  • 55
  • sorry for resuming an old post but i think that perspMatrix[8]=2/imgWidth*cx-1 and not 1-2/imgWidth*cx. My tought is based on equation (7) of this link: http://kgeorge.github.io/2014/03/08/calculating-opengl-perspective-matrix-from-opencv-intrinsic-matrix/ , can you give me a feedback about my comment? – Luca Feb 16 '15 at 11:54
  • The OpenGL projection matrix is a composition of several logical transforms (world coordinates to focal coordinates to image coordinates to normalized device coordinates), so the form of your final matrix really depends on what world coordinate frame you chose. I worked on that recently, so I'll edit my answer to make the associated world coordinate frame clearer. – BConic Feb 16 '15 at 12:55
  • 1
    In case you need a different convention for the input world coordinate frame, you'll have to derive the equation yourself. I found [Song Ho Ahn's page](http://www.songho.ca/opengl/gl_projectionmatrix.html) to be very helpful in that regard. – BConic Feb 16 '15 at 13:15
  • can someone explain how this projection matrix was generated? – techguy18985 Oct 05 '16 at 20:40
  • @user43053534 it is a bit tedious to do by hand, so I carefully derived the appropriate transformation equations and used a computer algebra system to simplify them. – BConic Oct 06 '16 at 17:44
  • I am getting almost the same projection matrix with you, but the only differences are on those parameters : 2*(cx/W)+1, 2*(cy/H)+1 The equation that I've used is : Projection matrix = NDC * Persp. projection, where NDC matrix is a matrix generated from the glOrtho function in OpenGL, and perspective projection matrix was generated by modifying a little bit the intrinsic parametres of the camera (preserving the Z-depth information and for correct clipping). – techguy18985 Oct 07 '16 at 11:31
  • @user43053534 Different coordinate conventions will lead to different results (see Luca's answer), so I cannot tell you why your matrix is different. But if your projection matrix works for you, then all is good :) – BConic Oct 07 '16 at 12:05
5

In relation to the AldurDisciple answer this is the formulation you need to use if the world coordinate frame is choosen with inverted z axe

inverted z axe

glMatrixMode(GL_PROJECTION); // Select The Projection Matrix
glLoadIdentity();            // Reset The Projection Matrix
GLdouble perspMatrix[16]={2*fx/w,0,0,0,0,2*fy/h,0,0,2*(cx/w)-1,2*(cy/h)-1,-(far+near)/(far-near),-1,0,0,-2*far*near/(far-near),0};
glMultMatrixd(perspMatrix);
Luca
  • 1,270
  • 1
  • 18
  • 34
  • sorry for commenting on an old post, but I would like to know the matrices that you've multiplied in order to understand how you generated the projection matrix. I saw some tutorials mentioned above but I still can't figure it out how the value -1 (I found that value to be +1) is generated in parameters 2*(cx/w)-1 and 2*(cy/h)-1. When I am using your projection matrix in my AR application the projection is correctly. – techguy18985 Oct 07 '16 at 14:40
  • why your reference frame is of LEFT hand? Isn't opengl define a RIGHT hand frame of reference? – zhangxaochen Jan 11 '17 at 07:18
  • Besides, if you invert the Z-axis as @AldurDisciple's answer, why don't you negate `2*(cx/w)-1,2*(cy/h)-1`? is that a mistake? – zhangxaochen Jan 11 '17 at 09:40
  • Amazing! This is exactly what I needed for a projection matrix in Three.js. – Eric Simonton Aug 27 '20 at 14:45
1

Open GL operates with a frustum which is related to perspective projection with some limits in depth referred as near and far values. Imagine a pyramid lying on its side - this is frustum. Another analogy is a projector beam that extends in its width and height with the distance - this is frustum too. So right, left, bottom, and top are your image coordinates while near and far are your depth limits with the near beint your focal plane. OpenGL will put Cx and Cy in the center of the image plane so you can skip them. The alternative and more natural way to specify frustum is based on viewing angle or field of view (50-60 deg is typical); the function you call is glPerspective() where you still have near and far but instead of sizes specify the angle and aspect ratio. Good luck.

Vlad
  • 4,425
  • 1
  • 30
  • 39
-1

You have cx, cy, fx, fy, width, height.
And you need left, right, top, bottom in your equations. You can calculate these value using this way. 

left = cx * near / -fx
top = cy * near / fy
right = -(width - cx) * near / -fx
bottom = -(height - cy) * near / fy
hao li
  • 367
  • 2
  • 13