Using openCV remap with float*

Question

I'm currently working on a project which involves motion compensation.

At this point I have the input image in cv::Mat input;

cv::resize( input, input_resized, cvSize(64,48), 12, 12, CV_INTER_AREA);
input_resized.convertTo(input_float, CV_32F); //this might be redundant while cv::rezise dst should be 32F.

Now the motion vectors are stored in a 2 channel cv::Mat motion and it's stored like this:

// w -image.width , h-image.height
    X0  Y0  X1   Y1   X2   Y2   ...  Xw-1  Yw-1      
    Xw  Yw  Xw+1 Yw+1 Xw+2 Yw+2 ...  X2w-1 Y2w-1
    ...
    X(h-1)w Y(h-1)w   ............. X(h-1)(w-1)Y(h-1)(w-1)

So if I use :

std::vector<cv::Mat> motionvect(2);
cv::split(motion, motionvect);

I will get X in motionvect(0) and Y in motionvect2(1).

Furthermore I've tried:

std::vector<cv::Mat> rgbChannels(3);
cv::split(input_float, rgbChannels);
cv::remap(rgbChannels[0], outChannels[0], motionvect[0], motionvect[1], CV_INTER_LINEAR, IPL_BORDER_CONSTANT, cvScalar(0,0, 0));
cv::remap(rgbChannels[1], outChannels[1], motionvect[0], motionvect[1], CV_INTER_LINEAR, IPL_BORDER_CONSTANT, cvScalar(0,0, 0));
cv::remap(rgbChannels[2], outChannels[2], motionvect[0], motionvect[1], CV_INTER_LINEAR, IPL_BORDER_CONSTANT, cvScalar(0,0, 0));

cv::merge(outChannels, input_with_motion);

The results I get after this are very strange and somebody told me it is because of the Y vector. Aparently if X is like this:

1  2  3  4  5
6  7  8  9  10
11 12 13 14 15

Then Y should be :

Is that correct? Sort of transpose matrix. If it is true, is there any function that will change the motiovect[2] to look like this or should I do it for each individual component?

Also, for the final step I would need to apply the remap at a different time in my process. Something like this:

Function1 ( (float*) input_float.data , (float*) motion.data, (unsigned char*) out.data );
   Function2 ( float* input, float* R, float* G, float* B);
   Function3 ( float* R, float* G, float* B, float* W, float* Z);

where R, G, B, W, Z are float* types and they are allocated with

 R = (float*) malloc (frame_size * sizeof(float) );

I need to apply motion compensation to float* A and float* B, but these are not allowed inputs to remap. They have to be InputArray and I can't find any method to convert float* to any accepted InputArray type.

I know I'm not very organized in thoughts, but I hope you'll understand at least one of my questions. 1. How to apply remap if I have the motion vectors stored in a cv::Mat like x0 y0 x1 y1 .. ? 2. Can I convert float* to an accepted input for cv::remap ?

Try the advice in [this article](http://www.aishack.in/2010/07/opencvs-c-interface/) in the "Accessing elements" section. If you figure it out, be kind and answer your own question here. — Chris O, Dec 05 '13 at 01:27
Thanks for the article, I've read it before, but I went through it again. I understand how to access the elements of cv::Mat. It is just that I don't really understand what map1 and map2 refer to exactly. At first I thought they are the motion vectors so I've set them to 0. motion_0_32FC2 =cv::Mat::zeros(img._size.height, cimg._size.width, CV_32FC2); Then I've split this cv::Mat in two vectors and I called the remap function with them. After that I saw that the output would take only the first element [position 0] from every channel and repeat it for the entire frame. — CRS, Dec 05 '13 at 20:41
Also if I set them to 1, the output would take the element on the position 1 on each channel and repeat it. That's why I thought maybe the content of map1 map2 should be the indexes so I've set them to look like 0 1 2 3 4 5 6 7 8 9 10 11 .. but now the output only has the first row and that one is different from the original. So the question would be how should map1, map2 look like in order to remap the picture on the same positions and the output to be identical. Then I will add the motion vectors to this content. — CRS, Dec 05 '13 at 20:48

score 2 · Answer 1 · answered Dec 06 '13 at 10:25

I've found this tutorial http://docs.opencv.org/doc/tutorials/imgproc/imgtrans/remap/remap.html very useful for understanding how to set map1, map2 parameters. In the end it's something like this:

for(int i=0; i< rows; i++){
        for(int j=0; j<col; j++)
            { 
             map_x.at<float>(i,j) = j + motion_vect.ptr<float>(i)[2*j+0];
             map_y.at<float>(i,j) = i + motion_vect.ptr<float>(i)[2*j+1];;
        }}

To have the output image as the original :

map_x.at<float>(i,j) = j ;
map_y.at<float>(i,j) = i ;

score 0 · Accepted Answer · answered Dec 05 '13 at 02:17

For your #2, you can do this to get float values from pointer into some Mat:

cv::Mat testMat = cv::Mat::zeros(2, 3, CV_32F);
std::cout << "(1, 2) elem = " << testMat.at<float>(1, 2) << std::endl;

float* pTestVal = new float;
*pTestVal = 3.14F;
testMat.at<float>(1, 2) = *pTestVal;
delete pTestVal;

std::cout << "(1, 2) elem = " << testMat.at<float>(1, 2) << std::endl;

Using openCV remap with float*

2 Answers2