How to use points / scene corners of homography as input for ROI in another function

Question

I want to perform object tracking after recognition, currently im able to recognize an object an draw lines around the object.I would now like to use these four points(scene corners) calculated in perspective transform as input for computing ROI and use it for tracking in subsequent frames.

My current implementation

JNIEXPORT jint JNICALL Java_org_opencv_samples_tutorial2_ObjReco_MatchFeatures(JNIEnv*, jobject, jlong addrGray12, jlong addrRgba12,jlong kalmanaddrRgb)
{
Mat& mGr12  = *(Mat*)addrGray12;
Mat& mRgb12 = *(Mat*)addrRgba12;
Mat H = findHomography( obj, scene, CV_RANSAC); // to find homography
perspectiveTransform( obj_corners, scene_corners, H);// obtain scene corners using perspective transform
setrectagnlepoint(scene_corners); // method to set the points 
}

JNIEXPORT void JNICALL Java_org_opencv_samples_tutorial2_LKTrack_LKTracker(JNIEnv*, jobject, jlong LKtrackgray1,jlong LKtracknew2,jlong LKTrackRgba1)
{

vector<Point2f> features;
vector<Point2f> RectRoi;
Mat RoiLK;
Rect rect;
Mat LKprev;
Mat& LKRgba1 = *(Mat*)LKTrackRgba1;
Mat& LKgray2 = *(Mat*)LKtrackgray1;
Mat regiongray;

LKcounter++;
if (LKcounter == 1) // set rect using points from previous method for the first time 
{
RectRoi=getrectagnlepoint(); // get points from previous method to set ROI
rect = Rect(RectRoi[0].x,RectRoi[0].y,(RectRoi[1].x-RectRoi[0].x),(RectRoi[3].y-RectRoi[0].y));
//rect = Rect(100,100,40,40);
}

LKgray2.copyTo(LKprev);
RoiLK= LKprev(rect);

    cv::goodFeaturesToTrack(RoiLK,features,6000,0.6,2,Mat(),2);

  // Update features to your ROI location
    for (i=0; i<features.size();i++)
    {
        features[i].x= features[i].x+RectRoi[0].x;
        features[i].y= features[i].y+RectRoi[0].y;
    }


// code to calculate optical flow

// code to predict new bounding box

}

currently i am facing assertion error when the points of the rectangle is skewed or if the points do not form a perfect rectangle.

OpenCV Error: Assertion failed (0 <= roi.x && 0 <= roi.width && roi.x + roi.width <= m.cols && 0 <= roi.y && 0 <= roi.height && roi.y + roi.height <= m.rows) in cv::Mat::Mat(const cv::Mat&, const Rect&), file /home/reports/ci/slave_desktop/50-SDK/opencv/modules/core/src/matrix.cpp, line 323

Question is, how do i make sure the points from perspective transform form a perfect rectangle and also is there a method to form ROI for a skewed rectangle?. Any guidance on this will be greatly helpful.

ROI are straight rectangles. "Make sure the points from perspective transform form a perfect rectangle", not sure what you need, a bounding box containing those 4 points or you want some test to check if the points form a rectangle? — Rui Marques, Apr 11 '14 at 11:21
I need bounding box containing four points. I dont know a method to get a perfect rectangle from a bounding box. I need to perform Lucas kanade tarcking with this bounding box. Im currently looking at this http://www.mathopenref.com/coordbounds.html . I will try to implement something similar, but please let me know if there is some better solution. — Darshan, Apr 11 '14 at 12:18

score 1 · Answer 1 · answered Apr 11 '14 at 12:00

1

it looks as if the transform maps the object points to points that are outside the scene region. you would have to check and verify the transformed points.

answered Apr 11 '14 at 12:00

dhanushka

10,492
2
37
47

Currently, this is not the problem .Problem is getting a perfect rectangle, but i will also have to look into your suggestion. – Darshan Apr 11 '14 at 12:23
1

to check if it's a rectangle, find the convex hull of the points, then find its minimum area rectangle. if the ratio between the convex-hull area and minimum-area-rectangle area is close to 1, you can safely assume the points form a perfect rectangle – dhanushka Apr 11 '14 at 13:03

score 1 · Accepted Answer · answered Apr 11 '14 at 13:03

OpenCV's ROI are always a straight rectangle.

You can check here how to create a bounding box for your 4 points - basically your 4 points are referred in that tutorial as a "contour".

Relevant code:

approxPolyDP( Mat(contour), contour_poly, 3, true );
boundRect = boundingRect( Mat(contour_poly) );

How to use points / scene corners of homography as input for ROI in another function

2 Answers2