1

I want to remove fisheye effect from image, so I need camera matrix and distortion cofficient. That's why I created cal.py file.

When I run that file it give me camera matrix and distortion efficient which I had put in undist.py file to undistort the image. now every time i got the same RMS,camera matrix,distortion coefficient.

but when i put this parameters in undist.py file that is give me blank image.. how to solve this problem?PLz help me...Thanks in advance.i have capture 8 image and one of the sample image of chessboard pattern.

snehal desai
  • 119
  • 5
  • 23
  • but why this matrices are wrong?where is the mistake in code?how i get right matrices value?i refer this link and commented that function[link](http://stackoverflow.com/questions/13478404/camera-calibration-cv-calibratecamera-and-cv-initundistortmap),if i uncommented that function it give me error so plz tell me how and where to call calibration fuction so it can give me right metrices? – snehal desai Apr 08 '16 at 09:46
  • Exactly, but obviously you need to solve the error inside the function for it to work... so, the output that you showed before is not valid... you should show the error instead, so it can be fixed. Your matrices are wrong because they are never assigned the correct camera_matrix, because you don't run the function that calls the calibrate function... – api55 Apr 08 '16 at 10:14
  • I think your calibration function is not even loading the images.... you get the `img_mask` from the program arguments, but then you overwrite it again.... probably that's a bug... you should print the image in every step to check it is doing it correctly.... you may check this [tutorial](http://opencv-python-tutroals.readthedocs.org/en/latest/py_tutorials/py_calib3d/py_calibration/py_calibration.html) in python opencv . Also, the link you referenced never showed how she solved the problem she had... – api55 Apr 08 '16 at 10:25
  • when i comment this:camera_matrix , dist_coefs = calibracion(),error is:OpenCV Error: Assertion failed (nimages > 0) in calibrateCamera, file /root/opencv-2.4.10/modules/calib3d/src/calibration.cpp, line 3415 Traceback (most recent call last): File "cal.py", line 58, in camera_matrix , dist_coefs = calibracion() File "cal.py", line 47, in calibracion rms, camera_matrix, dist_coefs, rvecs, tvecs = cv2.calibrateCamera(obj_points, img_points, (w, h)) cv2.error: /root/opencv-2.4.10/modules/calib3d/src/calibration.cpp:3415: error: (-215) nimages > 0 in function calibrateCamera – snehal desai Apr 08 '16 at 10:31
  • i don't know the solution of this..plz tell me what is the solution of this error..where i have to change in code? – snehal desai Apr 08 '16 at 10:32
  • it say that the number of images should be bigger than 0.... so that means that it is not receiving any imagepoints or object points.... you should check every step with your images to see what is going on... use the function `imshow` to see if the img is being load, and then if the lines are drawn correctly... check the tutorial i linked before – api55 Apr 08 '16 at 10:36
  • Thank u..i catch 20 images so it process on image and comes processing img6.jpg... chessboard not found processing img2.jpg... chessboard not found processing img14.jpg... chessboard not found processing img10.jpg... chessboard not found processing img9.jpg... chessboard not found processing img7.jpg... chessboard not found processing img1.jpg... chessboard not found processing img13.jpg...chessboard not found....and so on. and at the end comes same error...now what to do? ..link u provided i already tried that.it shows same error as above can u plz tell me step i have to followed for this. – snehal desai Apr 08 '16 at 10:58
  • you need to play with the function ´cv2.findChessboardCorners(img, pattern_size)´... you may have the wrong pattern size? you have `(9,6)`, are you sure you have 9 rows and 6 columns? also you may try adding a flag... but I think there is something else wrong, can you show one image? – api55 Apr 08 '16 at 11:59
  • Thank u..my image is captured image that i have captured using webcam in raspberry pi using fswebcam command..it has no chessboard..i don't have chessboard image becaz i want to capture image.. when i capture the image it is fisheye image so i want to remove that fisheye effect..so why required chessbord image? and if it is require than how i can get that chessboard image and than convert my captured image to nomal from fisheye image using that chessboard image. – snehal desai Apr 08 '16 at 12:15
  • thank u very much..now i have captured chessboard image from different angles and calculate camera matrix and distortion coefficient...camera matrix and distortion coefficient is also different every time when i run script but when i use this 2 parameter with other images(use this 2 parameter to undistort image in undist.py file as shown abve), image will blank,it means it is not distorted..so now where is problem? becaz no error coming,also msg displayed is processing image1.jpg...ok and so on. – snehal desai Apr 11 '16 at 05:59
  • You have to update the code and output so we can verify it is a valid camera matrix, distortion parameters etc... – api55 Apr 12 '16 at 08:23
  • Thank u api55..now i got the camera matrix and distortion coefficient same at every time when i run the cal.py, though the image is blank when run undist.py after putting this two parameter...and i also update my code and snapshot of output,which shows cameramatrix and distortion coefficient ..plz verify that and update me. – snehal desai Apr 12 '16 at 12:25
  • i have taken 640*480 resolution to capture image..so what i have to write?is camera matrix depend on resolution?if yes than how?and what i have to write in camera matrix to capture 640*480 resolution image? – snehal desai Apr 13 '16 at 04:17
  • yes it depends on the resolution, the cx and cy values depend on it, but they seem correct.... you should first try printing the values of the newcameramatrix to see if they are good, also, i think this step is not necessary `newcamera, roi = cv2.getOptimalNewCameraMatrix(K, d, (w,h), 0)` try commenting it and running it. I think it is not necessary because undistort will undistort your camera matrix and change some values.... and i think this function also undistorts it and change some camera values... – api55 Apr 13 '16 at 10:51
  • thank u...but when i comment this line and run it only remove some outer portion of image that's why it looks like it is zoom,but fisheye effect is still there...plz check the code once again and suggest me where is problem? – snehal desai Apr 13 '16 at 12:19
  • Maybe the images used where not enough and the distortion coefficients where not well calculated. For example, if most of the images have the same angle it may not work, it should be visible in several angles. Although your distortion coefficients are similar to those of a fisheye camera... so it should be ok.... can you show one example output of the undistorted image (and the input). Try to show something that should be rect, like a the chessboard pattern – api55 Apr 13 '16 at 13:19
  • ok..i have attach input and output image in the question..so plz see question and give me suggestion. – snehal desai Apr 14 '16 at 04:03
  • If you see the beam in your input output image, it is getting undistorted, but the outer parts of the image not.... I think you should try using more images to calibrate, that are closer to the edges of the image too... so this way the program should have more information about the distortion in different parts of the image, and give you an accurate distortion coeff. Try it out and tell me how it goes – api55 Apr 14 '16 at 08:38
  • thank u so much...now it's seems like it's working,now fisheye effect is removed but some outer portion of image is cropped so it seems like it it zoomed..i don't want to zoom it ..is there any solution for that or not?what i have to do for that? – snehal desai Apr 14 '16 at 10:35
  • it is probably due to this camera matrix you are passing to the undistort. `newcamera, roi = cv2.getOptimalNewCameraMatrix(K, d, (w,h), 0)` and `newimg = cv2.undistort(img, K, d, None, newcamera)` So you get and optimal one, and then pass it to the undistort... with this you may zoom or skew or change the image... you can pass an empty variable and see what happens – api55 Apr 14 '16 at 14:13
  • one more problem..i changed the webcam and take 17 different chessboard pattern image using that then run cal.py and this give me below value of camera matrix and dis coefficient: RMS: 0.835540866152 camera matrix: [[ 275.20841672 0. 308.82245907] [ 0. 273.87788679 242.18360662] [ 0. 0. 1. ]] distortion coefficients: [-0.39464661 0.23664665 -0.00629918 -0.00109528 -0.10138152] i put this value in undist.py file so it give me input and output image shown in updated question(see updated image in question).is calculated parameter are wrong?why is that happening?what changes i have to do?plz help – snehal desai Apr 15 '16 at 04:59

1 Answers1

2

I will do an answer to summarize all the problems solved in the comments (this way the future readers do not have to read all of them).

  1. Follow the tutorial from here to have a better understanding of what you should do and which functions to use.

  2. Check that your camera matrix have the correct form:

    [ f_x  s    c_x
  0    f_y  c_y
  0    0    1  ]

    the s is a skew value that I think in opencv it always gives 0.

  3. Make sure in every step that the images are loading correctly and that they are doing exactly what is intended (use imshow function to debug)

  4. This part of your code

    newcamera, roi = cv2.getOptimalNewCameraMatrix(K, d, (w,h), 0)
newimg = cv2.undistort(img, K, d, None, newcamera)

is faulty. Why, well from the opencv documentation we have that:

cv2.getOptimalNewCameraMatrix(cameraMatrix, distCoeffs, imageSize, alpha[, newImgSize[, centerPrincipalPoint]]) → retval, validPixROI

and

The function computes and returns the optimal new camera matrix based on the free scaling parameter. By varying this parameter, you may retrieve only sensible pixels alpha=0 , keep all the original image pixels if there is valuable information in the corners alpha=1 , or get something in between. When alpha>0 , the undistortion result is likely to have some black pixels corresponding to “virtual” pixels outside of the captured distorted image. The original camera matrix, distortion coefficients, the computed new camera matrix, and newImageSize should be passed to initUndistortRectifyMap() to produce the maps for remap() .

This means that the new camera matrix also gives you a new valid size!! To be more specifically:

the computed new camera matrix, and newImageSize should be passed to initUndistortRectifyMap()

But the undistort function does the initUndistortRectifyMap automatically.... and it doesn't have a way to pass the newImageSize from this function. So basically you have 2 options.

  1. Use your newcamera matrix, but instead of doing undistort, you should do everything manually.... this means that you must do initUndistortRectifyMap and remap using the new sizes and new camera matrix.
  2. Use the original camera matrix obtained in the calibrateCamera function. This way it will not have this zoom effect, but you may have some extra black pixels representing the areas that you may not see due to rectification.

If not, it will always give you this zoom effect, because it will not show the invalid pixels (black pixels) of the undistorted areas.

api55
  • 11,070
  • 4
  • 41
  • 57
  • Thank u so much..all problem are solved...but it still display cut image from all side while convert normal image..i think u told me about this but i don't understood.. plz tell me again in detail why this is happning?and how we get full image without cut or zoom it?is there any way to do this?i want full image not cut image. – snehal desai Apr 19 '16 at 04:42
  • i also heard that while converting fisheye image to normal it generates 5 images from one(upper,below,front,left,right)..is it true?if yes than how we get all these 5 images?and other than front part all images are in fisheye shape or they also converted in normal image with fornt image while converting? – snehal desai Apr 19 '16 at 04:42
  • @snehaldesai As far as I know it is only generating 1 image. What opencv does is generate a mapping function. This will tell the position of each pixel in the new image. When you undistort the image, there are pixels that where not found in the distorted image, so they will appear black.... the get optimalNewCameraMatrix, tries to cut this pixels and resize the image to fit the size. – api55 Apr 19 '16 at 08:38