8

Good day guys,

I am trying to simulate and image with lens barrel distortion. I create a virtual chessboard (only the corners) and then project it onto my image plane using OpenCV. The idea is to project these points with known distortion coefficients, and then attempt a lens distortion calibration (with calibrateCamera), and see if the same coefficients are obtained.

My question is about the projectPoints function which takes distCoeffs as an input. Are these coefficients the same that must be used to undistort an image (output of calibrateCamera)? This means the function will have to calculate the inverse of that operation. Or, does it use those coefficients to distort the object points directly? Meaning that the will not correlate at all at the output of e.g. calibrateCamera.

I ask, because I noticed my simulation does pincushion distortion when I expect barrel, and vica versa. Which seems that the distortion does the opposite of what I think it does.

The minimal working code that I used to simulate the image (in Python):

# The distortion matrix that I vary
distortion = np.array([0.3, 0.001, 0.0, 0.0, 0.01])

# Generate Grid of Object Points
grid_size, square_size = [20, 20], 0.2
object_points = np.zeros([grid_size[0] * grid_size[1], 3])
mx, my = [(grid_size[0] - 1) * square_size / 2, (grid_size[1] - 1) * square_size / 2]
for i in range(grid_size[0]):
    for j in range(grid_size[1]):
        object_points[i * grid_size[0] + j] = [i * square_size - mx, j * square_size - my, 0]

# Setup the camera information
f, p = [5e-3, 120e-8]
intrinsic = np.array([[f/p, 0, 0], [0, f/p, 0], [0, 0, 1]])
rvec = np.array([0.0, 0.0, 0.0])
tvec = np.array([0.0, 0.0, 3.0])

# Project the points
image_points, jacobian = cv2.projectPoints(object_points, rvec, tvec, intrinsic, distortion)

# Plot the points (using PyPlot)
plt.scatter(*zip(*image_points[:, 0, :]), marker='.')
plt.axis('equal')
plt.xlim((-4000, 4000))
plt.ylim((-4000, 4000))
plt.grid()
plt.show()

Additional Explanation:

My query is about the wrong distortion type being created. If I use a positive distortion matrix, then I would expect barrel distortion, according to this website, which states:

The next figure shows two common types of radial distortion: barrel distortion (typically k_1 > 0) and pincushion distortion (typically k_1 < 0).

To test this I used the following positive distortion matrix (as in the code above), and it created pincushion distortion.

distortion = np.array([0.3, 0.001, 0.0, 0.0, 0.01])

Expected Barrel Distortion, but got Pincushion

In this image I clearly created pincushion distortion, and not barrel distortion.

Similarly, if I make my distortion coefficient negative, which should result in pincushion distortion, I get the following:

distortion = -np.array([0.3, 0.001, 0.0, 0.0, 0.01])

Expected Pincushion distortion, got barrel distortion

Does this mean that the distortion matrix is applied negatively if you use the projectPoints function?

Thank you for your help!

Bhargav Rao
  • 50,140
  • 28
  • 121
  • 140
Hein Wessels
  • 937
  • 5
  • 15
  • I guess this is just a matter of the pinhole model used by opencv and the one you are used to. you can think it as the `physical` pinhole where the image is projected inverted. Like (https://en.wikipedia.org/wiki/File:Pinhole-camera.svg) But many times, we use a model where the projection is in front and not inverted (https://openmvg.readthedocs.io/en/latest/_images/pinholeCamera.png). These 2 models are quite the same, the difference is only in the signal of the dimensions. – Fred Guth Aug 18 '18 at 22:09
  • @Fred, thank you for your comment! I am aware of the difference you talk about, but that would create a inverted `x` and `y` axis, which is not what I'm struggling with. Instead the wrong distortion type is created. I updated my question to make this more clear. – Hein Wessels Aug 20 '18 at 08:00

1 Answers1

3

It seems that the distCoeffs in projectPoints are of the same type that calibrateCamera outputs. I ran a more thorough simulation where I added distortion using projectPoints, and then estimated it again using calibrateCamera, and it gave me the same distortion matrix.

E.g with the following code

distortion_given = np.array([[0.2, 0.01,  0.0, 0.0, 0.01]])# Note the negative sign
... = projectPoints(... , distortion_given , ...)
... , distortion_estimated , ... = calibrateCamera (...)
print (distortion_given)
print (distortion_estimated)

which created the following projected image with Barrel distortion

enter image description here

Resulted in:

distortion_given = [[-0.2, -0.01,  0.0, 0.0, -0.01]]
distortion_estimated = [[-0.19999985 -0.01000031  0.          0.         -0.00999981]]

This means that when this website I mentioned in the question should put more emphasis on the word typically, as I observed the opposite.

barrel distortion (typically k_1 > 0) and pincushion distortion (typically k_1 < 0).

Hein Wessels
  • 937
  • 5
  • 15