How to apply fisheye effect on a normal image using opencv-fisheye module in python

Question

I am trying to apply fisheye effect on a normal image using opencv-fisheye module.

For converting a fisheye image to normal image, 'cv2.fisheye.undistortimage' api can be used. I am not sure how to get the fisheye image with the help of 'cv2.fisheye.distortPoints' function. Any working sample code/algorithm is much appreciated.

I do not see any module for distorting an image, only points or only undistorting an image at https://docs.opencv.org/4.1.1/db/d58/group__calib3d__fisheye.html#ga167df4b00a6fd55287ba829fbf9913b9 — fmw42, May 13 '20 at 04:05
cv2.undistort(img, camera_matrix, dist_coefs, None, newcameramtx) try swapping newcameramtx and camera_matrix — Alex Alex, May 13 '20 at 05:53

score 1 · Answer 1 · answered Aug 18 '23 at 00:09

Using cv2.fisheye.distortPoints() we may obtain the coordinates of the destination (distorted) image from those of the source image.

As described here and here, "Note that the function assumes the camera matrix of the undistorted points to be identity. This means if you want to transform back points undistorted with undistortPoints() you have to multiply them with P−1".

The function cv2.fisheye.undistortPoints() accepts normalized coordinates as an input, hence we need to multiply the original indices by the inverse of the camera intrinsic matrix, as shown below.

The matrix camera intrinsic K and the distortion coefficients d are needed to be obtained through the camera calibration:

The input parrot image can be obtained from here.

import matplotlib.pylab as plt
import cv2

K = np.array( [[338.37324094,0,319.5],[0,339.059099,239.5],[0,0,1]], dtype=np.float32)   # camera intrinsic parameters
d = np.array([0.17149, -0.27191, 0.25787, -0.08054], dtype=np.float32) # k1, k2, k3, k4 - distortion coefficients

def apply_fisheye_effect(img, K, d):

   indices = np.array(np.meshgrid(range(img.shape[0]), range(img.shape[1]))).T \
            .reshape(np.prod(img.shape[:2]), -1).astype(np.float32)

   Kinv = np.linalg.inv(K)
   indices1 = np.zeros_like(indices, dtype=np.float32)
   for i in range(len(indices)):
      x, y = indices[i]
      indices1[i] = (Kinv @ np.array([[x], [y], [1]])).squeeze()[:2]
   indices1 = indices1[np.newaxis, :, :]

   in_indices = cv2.fisheye.distortPoints(indices1, K, d)
   indices, in_indices = indices.squeeze(), in_indices.squeeze()

   distorted_img = np.zeros_like(img)
   for i in range(len(indices)):
      x, y = indices[i]
      ix, iy = in_indices[i]
      if (ix < img.shape[0]) and (iy < img.shape[1]):
         distorted_img[int(ix),int(iy)] = img[int(x),int(y)]

   return distorted_img

K = np.array( [[338.37324094,0,319.5],[0,339.059099,239.5],[0,0,1]],dtype=np.float32) # camera intrinsic params
d = np.array([0.17149, -0.27191, 0.25787, -0.08054],dtype=np.float32) # k1, k2, k3, k4 - distortion coefficients

img = plt.imread('parrot.jpg')
img = img / img.max()
distorted_img = apply_fisheye_effect(img, K, d)

plt.figure(figsize=(15,7))
plt.subplot(121), plt.imshow(img, aspect='auto'), plt.axis('off'), plt.title('original', size=20)
plt.subplot(122), plt.imshow(distorted_img, aspect='auto'), plt.axis('off'), plt.title('distorted', size=20)
plt.show()

Note that the above implementation can be made much faster by sampling a few points (instead of all the points) and using interpolation (with scipy, e.g.).

How to apply fisheye effect on a normal image using opencv-fisheye module in python

1 Answers1