2

I wanted to rectify a stereo image pair coming from two different modalities (visual and thermal). I calibrated both cameras using 

[cameraParams,imagesUsed,estimationErrors] = estimateCameraParameters(imagePoints,worldPoints);

giving me a stereoParameter object, since imagePoints contains matching checkerboard points for both modalities.

To rectify, I used the following call:

[J1,J2] = rectifyStereoImages(I1,I2, cameraParamsStereo);

where I1 is a visual image and I2 a thermal one. Unfortunately, that gives me the error:

Error using rectifyStereoImages>parseInputs (line 106) Inputs must be of the same size and data types.

Error in rectifyStereoImages (line 96) [I1, I2, interp, outputView, fillValues] = parseInputs(...

The resolutions of both images are quite different (2048x1088 for visual, 384x288 for thermal). From my undertanding however, rectification in principle should still work, since it is done similarly in this paper. Honestly however, I am not sure how...

Question: Is there a way in MATLAB to rectify images of different sizes? If not, is there an algorithm to do so that can be easily implemented?

rayryeng
  • 102,964
  • 22
  • 184
  • 193
user1809923
  • 1,235
  • 3
  • 12
  • 27
  • have you tried making your thermal image the same size than the optical ? add empty row and columns to the right and to the bottom. – gregswiss Sep 16 '15 at 06:50
  • @gregswiss: With empty I suppose you mean zero values. I tried that now, but this gives me another error "Index exceeds matrix dimensions" in "Error in vision.internal.calibration.CameraParametersImpl/getValidBounds (line 860)" – user1809923 Sep 16 '15 at 08:16

1 Answers1

0

Unfortunately, the rectifyStereoImages function requires the two images to have the same size. This is a limitation of the implementation, not the algorithm.

One thing you can do is undistort the images using the undistortImage function, find matching points (e.g. using matchFeatures), and then use the triangulate function to get a sparse 3-D reconstruction.

You can also try making the two images the same size, by padding the smaller image with zeros. You would need to pad the calibration images before calibration, so that everything is consistent.

Dima
  • 38,860
  • 14
  • 75
  • 115
  • sorry for my late reaction. I really want to rectify the images, so your first way does not work unfortunately. As for your second way I am confused: if I just pad the images with zeros on the right and bottom the corner points for the checkerboards will not change - so estimateCameraParameters(imagePoints,worldPoints) should give me the same results as before the padding? – user1809923 Sep 29 '15 at 07:08
  • That's a thought... Try it. – Dima Sep 29 '15 at 12:14
  • I did. I get the same error as before, stated in my comment from the 16th of September above. The corner points of the padded image do not change and estimateCameraParameters(imagePoints,worldPoints) does not get any information that the images changed whatsoever (were padded). Even if I use the stereo calibration app, calibrate (which now works with the padded images) and then click on "Show rectified" in an example, I get the same error. – user1809923 Sep 29 '15 at 13:29
  • Most likely that's because the data types are different. IR images are typically 16 bit. Convert the data type of the one image to match the other. Use `im2uint16`, or convert both to double or single using `im2double` or `im2single`. – Dima Sep 29 '15 at 13:55
  • hmm..does not seem to be the problem, both are of type uint8 when loaded with imread. Should I open a new question for this or edit my question above with a more detailed error description? Unfortunately, I probably cannot provide my image sets... – user1809923 Sep 29 '15 at 14:24
  • Is the non-IR image RGB? Then the size is different: M x N x 3 vs. M x N x 1. If that's the case, use `rgb2gray` on the RGB image. – Dima Sep 29 '15 at 14:33
  • Ok, are you sure that the two images are *exactly* the same size and data type? Can you please double-check? Can it be that you have a different data type after padding? Also, which version of matlab are you using? – Dima Sep 29 '15 at 15:13
  • Yes, I am sure. Also, if I input two images of different size or type MATLAB will output a specific error. Moreover, if I input the same image into rectifyStereoImages twice, it will also give me the error mentioned above. If you want, I can provide you with "blacked-out" versions of the images :) Oh, and I use Matlab R2015a – user1809923 Sep 30 '15 at 07:01
  • What specific error? If you are seeing "Inputs must be of the same size and data types.", then the images either have a different size or a different data type. Please check that just before the call to `rectifyStereoImages`. If you are seeing some other error message, please let me know. – Dima Sep 30 '15 at 15:00