5

I'm developing an augmented reality application - a virtual try-on, using OpenCV + OpenGL + QtCreator - and I'm stuck now at calibrating the camera. I found a lot of resources about the calibration process in OpenCV using the chessboard pattern, but I need to implement some sort of self-calibration, so that's not helpful. I know it can be done, but didn't really find anything useful. I found this research http://www.eidelen.ch/thesis/MscThesisV1.0.pdf in which a self-calibration process is described (chapter 4), but I'm not sure if that's the way to go. What I want to achieve can be seen at http://www.ray-ban.com/usa/virtual-mirror. I just want to know how do they calibrate.

joanna
  • 743
  • 3
  • 13
  • 27
  • Why not use a classic calibration procedure with chessboard pattern? – BConic May 03 '14 at 17:39
  • I need to calibrate in real-time, not force the user to do that. – joanna May 04 '14 at 14:19
  • afaik "bundle adjustment" solves intrinsic and extrinsic camera parameters for some number of images and an initial guess. As a guess for the principal point you should choose the image center. For the rest I'm not sure ;) – Micka May 05 '14 at 09:05

2 Answers2

1

For camera calibration you need to know a set of real coordinates in the world. The chessboard gives you that since you know the size and shape of the squares, so you can correlate pixel locations with measurements in the real world.

You'll see that in Schneider's thesis he uses a 3D tracking unit (Figure 3.1) to give him the real-world coordinates of the points. One he has those, it's a similar problem to the chessboard.

In the virtual mirror example, I don't know but I'd guess that they are using a face detection system, and thus do not need a calibrated image. Something like: http://www.vision.caltech.edu/html-files/EE148-2005-Spring/pprs/viola04ijcv.pdf

For your system that might make more sense. Lots of people do face detection in OpenCV, so there's plenty around on that. You might start here: http://docs.opencv.org/trunk/modules/contrib/doc/facerec/facerec_tutorial.html

abarry
  • 425
  • 3
  • 16
  • I actually did work with the Haar classifiers in OpenCV (they use the Viola-Jones method you mentioned) for face/eyes detection and it's pretty easy & straightforward. Got good results too. But I don't understand how can I use for calibration the rectangles I obtain as a result. I understand that calibration can be done in real-time if I get at least 4 real-world points for which I know the corresponding 2D points. But how can I obtain such points in my situation ? Also, what do you mean by 'thus do not need a calibrated image' ? – joanna May 04 '14 at 14:40
  • What exactly are you trying to do? If you're trying to make an application like the virtual mirror, you probably don't need a calibrated camera. For example, you might need to know the width of the person's face (in pixels) to scale a picture of glasses the right amount. I'm guessing that that is what they are doing on the site. – abarry May 05 '14 at 02:28
  • Every AR example I've seen similar to what I'm trying to achieve uses OpenCV's solvePnP to obtain the rotation & translation vectors in the camera coordinate system. But for this you need the intrinsic matrix & distortion coefficients, obtained by calibrating the camera. So I don't know how to get pass this. I found this example https://www.youtube.com/watch?v=bV-jAnQ-tvw with source code provided, but I don't know how to apply it to suit my needs, as I don't know how to calibrate. – joanna May 05 '14 at 05:50
  • If you want to use solvePnP, you need that calibration -- there's no getting around that. You can't get a calibration without at some point knowing coordinates in the real world. An alternative is to attempt to write your application without requiring solvePnP, but without knowing more about what you are trying to do, I can't say if that's a possibility. – abarry May 05 '14 at 15:42
0

Camera self-calibration methods exist. They all use an assumption or another like rigidity. Take a look at this paper. There is a summary here.

globalcaos
  • 7
  • 5
  • 1
    Update: I explained my findings and re-formulated a more precise question [here](https://stackoverflow.com/questions/45247683/easy-monocular-camera-self-calibration-algorithm) – globalcaos Jul 21 '17 at 23:34