0

When I run projectPoints function using java code below with the most simple inputs I get what I believe is incorrect answer. Did I miss something?

I get this answer

(x,y,z)= [{0.0, 1.0, 1.0}, {0.0, 1.0, 5.0}, {0.0, 1.0, 10.0}]
(u,v)= [{0.0, 78.5398178100586}, {0.0, 19.73955535888672}, {0.0, 9.966865539550781}]

should be (u,v)=[{0.0, 100.0}, {0.0, 20.0}, {0.0, 10.0}]

Below is the code that calls projectPoint or alternately a function I wrote that works correctly (without camera distortion). 

public static void main(String[] args) {
    setDllLibraryPath("C:/aaa_eric/code/lib/x64");
    System.loadLibrary(Core.NATIVE_LIBRARY_NAME);

    MatOfPoint3f objectPts3f = new MatOfPoint3f(
            new Point3(0.0,1.0, 1.0),
            new Point3(0.0,1.0, 5.0),
            new Point3(0.0,1.0,10.0));
    MatOfPoint2f imagePts2f = new MatOfPoint2f();
    Mat rVec =  Mat.zeros(3,1, CvType.CV_64F);
    Mat tVec = Mat.zeros(3,1, CvType.CV_64F);

    //camera matrix, no distortion
    Mat kMat = Mat.zeros(3, 3, CvType.CV_64F);
    kMat.put(0, 0,
            100.0, 0.0, 0.0,
            0.0, 100.0, 0.0,
            0.0, 0.0, 1.0);
    Mat dMat = Mat.zeros(4, 1, CvType.CV_64F);

    //this one is broken
    Calib3d.projectPoints(objectPts3f, imagePts2f, rVec, tVec, kMat, dMat);
    //this one works
    ppWorks(objectPts3f, imagePts2f, rVec, tVec, kMat, dMat);

    System.out.println(objectPts3f.toList());
    System.out.println(imagePts2f.toList());
}

static void ppWorks(MatOfPoint3f objPt3f,MatOfPoint2f imgPt2f,Mat rVec,Mat tVec,Mat kMat, Mat dMat) {
    double[] T = new double[3]; tVec.get(0,0,T);
    double[] K = new double[9]; kMat.get(0,0,K);
    double[] D = new double[4]; dMat.get(0,0,D);

    Mat rMat = Mat.zeros(3,3,CvType.CV_64F);
    Calib3d.Rodrigues(rVec, rMat);
    double[] R = new double[9]; rMat.get(0,0,R);

    Point3[] xf=objPt3f.toArray();
    List<Point> y=new ArrayList<>();
    for (int i = 0; i < objPt3f.total(); i++) {
        Point3 Xi =  xf[i];
        Point3 Y = new Point3(
                R[0]*Xi.x + R[1]*Xi.y + R[2]*Xi.z + T[0]*1.0,
                R[3]*Xi.x + R[4]*Xi.y + R[5]*Xi.z + T[1]*1.0,
                R[6]*Xi.x + R[7]*Xi.y + R[8]*Xi.z + T[2]*1.0
                );

        y.add(new Point(K[0]*Y.x/Y.z + K[2], K[4]*Y.y/Y.z + K[5]));
    }
    imgPt2f.fromList(y);
}
Dmitrii Z.
  • 2,287
  • 3
  • 19
  • 29
J.E.Tkaczyk
  • 557
  • 1
  • 8
  • 19
  • BTW, I implemented similar code in python and the openCV::projectPoints function works there. I also posted this question on the openCV forum in hopes of finding the answer. I'll report back if answer is found there. – J.E.Tkaczyk Feb 17 '18 at 22:19

1 Answers1

1

Also faced the same issue (Java, OpenCV 3.4.1). Looks like the problem is that Calib3d class has mixed projectPoints methods for "standard" and fisheye camera models with same method names.

Your custom function had worked for me, and then I simply tried similar OpenCV method but with different signature and it worked:

// distortion coefficients have to be MatOfDouble in this signature
MatOfDouble dMat = new MatOfDouble(0,0,0,0,0);
// this one works fine
Calib3d.projectPoints(objectPts3f, rVec, tVec, kMat, dMat, imagePts2f, new Mat(), 0);

Note, that looks like OpenCV 4.0.0 had fixed the issue by prefixing fisheye method names (javadocs).

Sergeant
  • 26
  • 1