I am trying to recode feature matching and homography using mexopencv .Mexopencv ports OpenCV vision toolbox into Matlab .
My code in Matlab using OpenCV toolbox:
function hello
close all;clear all;
disp('Feature matching demo, press key when done');
boxImage = imread('D:/pic/500_1.jpg');
boxImage = rgb2gray(boxImage);
[boxPoints,boxFeatures] = cv.ORB(boxImage);
sceneImage = imread('D:/pic/100_1.jpg');
sceneImage = rgb2gray(sceneImage);
[scenePoints,sceneFeatures] = cv.ORB(sceneImage);
if (isempty(scenePoints)|| isempty(boxPoints))
return;
end;
matcher = cv.DescriptorMatcher('BruteForce');
matches = matcher.match(boxFeatures,sceneFeatures);
%Box contains pixels coordinates where there are matches
box = [boxPoints([matches(2:end).queryIdx]).pt];
%Scene contains pixels coordinates where there are matches
scene = [scenePoints([matches(2:end).trainIdx]).pt];
%Please refer to http://stackoverflow.com/questions/4682927/matlab-using-mat2cell
%Box arrays contains coordinates the form [ (x1,y1), (x2,y2) ...]
%after applying mat2cell function
[nRows, nCols] = size(box);
nSubCols = 2;
box = mat2cell(box,nRows,nSubCols.*ones(1,nCols/nSubCols));
%Scene arrays contains coordinates the form [ (x1,y1), (x2,y2) ...]
%after applying mat2cell function
[nRows, nCols] = size(scene);
nSubCols = 2;
scene = mat2cell(scene,nRows,nSubCols.*ones(1,nCols/nSubCols));
%Finding homography between box and scene
H = cv.findHomography(box,scene);
boxCorners = [1, 1;... % top-left
size(boxImage, 2), 1;... % top-right
size(boxImage, 2), size(boxImage, 1);... % bottom-right
1, size(boxImage, 1)];
%Fine until this point , problem starts with perspectiveTransform
sceneCorners= cv.perspectiveTransform(boxCorners,H);
end
The error:
Error using cv.perspectiveTransform
Unexpected Standard exception from MEX file.
What()
is:C:\slave\builds\WinInstallerMegaPack\src\opencv\modules\core\src\matmul.cpp:1926:
error: (-215) scn + 1 == m.cols && (depth == CV_32F || depth == CV_64F)
..
Error in hello (line 58)
sceneCorners= cv.perspectiveTransform(boxCorners,H);
The problem starts from checking out the perspectiveTranform(boxCorners, H), until finding homography it was fine . Also note that while calculating the matching coordinates from the sample and the scene , I indexed from 2:end, box = [boxPoints([matches(2:end).queryIdx]).pt], since accessing the queryIdx of the 1st element would yield the zeroth position that couldn't be accessed . However , I think , this would not be a problem . Anyhow , I am looking forward for an answer to my solution . Thanks.
PS:This is an edited version of my original post here . The solution I received below ,was not adequate enough , and the bug kept recurring .
2nd Update:
According to @Amro , I have updated my code ,below . The inliers gives good response , however the coordinates for calculating perspective transform somehow got twisted.
function hello
close all; clear all; clc;
disp('Feature matching with ORB');
%Feature detector and extractor for object
imgObj = imread('D:/pic/box.png');
%boxImage = rgb2gray(boxImage);
[keyObj,featObj] = cv.ORB(imgObj);
%Feature detector and extractor for scene
imgScene = imread('D:/pic/box_in_scene.png');
%sceneImage = rgb2gray(sceneImage);
[keyScene,featScene] = cv.ORB(imgScene);
if (isempty(keyScene)|| isempty(keyObj))
return;
end;
matcher = cv.DescriptorMatcher('BruteForce-HammingLUT');
m = matcher.match(featObj,featScene);
%im_matches = cv.drawMatches(boxImage, boxPoints, sceneImage, scenePoints,m);
% extract keypoints from the filtered matches
% (C zero-based vs. MATLAB one-based indexing)
ptsObj = cat(1, keyObj([m.queryIdx]+1).pt);
ptsObj = num2cell(ptsObj, 2);
ptsScene = cat(1, keyScene([m.trainIdx]+1).pt);
ptsScene = num2cell(ptsScene, 2);
% compute homography
[H,inliers] = cv.findHomography(ptsObj, ptsScene, 'Method','Ransac');
% remove outliers reported by RANSAC
inliers = logical(inliers);
m = m(inliers);
% show the final matches
imgMatches = cv.drawMatches(imgObj, keyObj, imgScene, keyScene, m, ...
'NotDrawSinglePoints',true);
imshow(imgMatches);
% apply the homography to the corner points of the box
[h,w] = size(imgObj);
corners = permute([0 0; w 0; w h; 0 h], [3 1 2]);
p = cv.perspectiveTransform(corners, H)
p = permute(p, [2 3 1])
p = bsxfun(@plus, p, [size(imgObj,2) 0]);
% draw lines between the transformed corners (the mapped object)
opts = {'Color',[0 255 0], 'Thickness',4};
imgMatches = cv.line(imgMatches, p(1,:), p(2,:), opts{:});
imgMatches = cv.line(imgMatches, p(2,:), p(3,:), opts{:});
imgMatches = cv.line(imgMatches, p(3,:), p(4,:), opts{:});
imgMatches = cv.line(imgMatches, p(4,:), p(1,:), opts{:});
imshow(imgMatches)
title('Matches & Object detection')
end
The output is fine , however , the perspectiveTranform is not giving the right coordinates apropos to the problem .
My output thus far :
3rd Update:
I have got all of the code running and fine with the homography . However , a corner case is bugging me really hard .
If I do imgObj = imread('D:/pic/box.png') and imgScene = imread('D:/pic/box_in_scene.png') , I get the homography rectangle good and fine , however , when I do imgScene = imread('D:/pic/box.png') , i.e the object and scene are the same , I get this error -
Error using cv.findHomography
Unexpected Standard exception from MEX file.
What()
is:C:\slave\builds\WinInstallerMegaPack\src\opencv\modules\calib3d\src\fundam.cpp:1074:
error: (-215) npoints >= 0 && points2.checkVector(2) == npoints && points1.type() ==
points2.type()
..
Error in hello (line 37)
[H,inliers] = cv.findHomography(ptsObj, ptsScene, 'Method','Ransac');
Now , I have came across this error in the past , this happens when the number of ptsObj or ptsScene is low , e.g, when the scene is nothing but a white/black screen , keypoints of that scene is zero . In this particular problem there is ample amount of ptsObj and ptsScene. Where can the problem lie . I have tested this code using SURFan the same error is resurfacing .
