Showing result of template matching

Question

I ran into another problem while making template matching in Halide (original link with resolved issue: output shifted in template matching)

Now I'm trying to draw a rectangle at the position with the lowest score (which indicates the best match).

template matching part:

Image<float> source = load_image("C:\\Users\\Admin\\Desktop\\templateMatchingOpenCV\\clip\\clip2.png");
Image<float> templ = load_image("C:\\Users\\Admin\\Desktop\\templateMatchingOpenCV\\clip\\object3.png");
Var x, y, xt, yt, x_outer, y_outer, x_inner, y_inner, tile_index;

RDom r(0, templ.width(), 0, templ.height());

Func limit, comparesqdiff, comparesqdiffnorm, compareccorr;
limit = BoundaryConditions::constant_exterior(source, 1.0f);

Expr function = sum(Halide::pow(templ(r.x, r.y) - limit(x + r.x, y + r.y), 2)) / (templ.width()*templ.height());
comparesqdiff(x, y) = sum(Halide::pow(templ(r.x, r.y) - limit(x + r.x, y + r.y), 2)) / (templ.width()*templ.height());
Image<float> outputsqdiff;

comparesqdiff.tile(x, y, x_outer, y_outer, x_inner, y_inner, 64,64).fuse(x_outer, y_outer, tile_index).vectorize(x_inner).unroll(y_inner).parallel(tile_index);

comparesqdiff.compile_jit();

Now it's clear to me that the function I should be using to find the position of the lowest score is argmin but i don't quite understand how it's used. Also I am aware that the drawing method would cover everything from that's below and right of the pixel but I didn't get to that part yet.

Drawing the rectangle part:

RDom wholeImage(0, source.width() - templ.width(), 0, source.height() - templ.height());

Tuple coords = argmin(r, function, "argmin");

Func show;

show(x, y) = select(x >= coords[0] && y >= coords[1] && x <= coords[0] + templ.width() && y <= coords[1] + templ.height(), 0, limit(x, y));

Image<float> test(source.width(), source.height());

test = show.realize(source.width(), source.height());

Thank you in advance.

Khouri Giordano · Accepted Answer · 2016-08-24T21:13:58.277

1

The argmin reduction iterates over your RDom, compares the values and keeps the location of the lowest value and the value.

Halide::RDom matchDom
    ( 0, templ.width ()
    , 0, templ.height()
    );
Halide::RDom searchDom
    ( 0, source.width () - templ.width ()
    , 0, source.height() - templ.height()
    );
Halide::Expr score = Halide::sum
    ( matchDom 
    , Halide::pow
        ( templ( matchDom.x, matchDom.y )
        - limit( searchDom.x + matchDom.x
               , searchDom.y + matchDom.y)
        , 2
        )
    )
    / ( templ.width() * templ.height() );
Halide::Tuple searchBest = Halide::argmin( searchDom, score );
Halide::Func best;
best(_) = searchBest(_);

Then you can call best.realize() to get a Halide::Realization. That realization will contain 3 buffers, each of a single value: the x coordinate of the lowest value, the y coordinate of the lowest value and the lowest value.

Halide is not the best tool for drawing geometric shapes. For my money, it would just be easier to write pixels into the image with a for loop.

ASKER'S EDIT: added marking the best result which uses the answer's method for finding best score

Realization re = best.realize();

Func draw("draw");

Image<int> x_coordinate(re[0]);
Image<int> y_coordinate(re[1]);
Image<float> s(re[2]);

draw(x,y) = select(x == x_coordinate(0, 0) || y == y_coordinate(0,0) || x == (x_coordinate(0,0) + templ.width()) || y ==  (y_coordinate(0,0) + templ.height()), 0.0f, source(x,y));

Image<float> drawTest;

drawTest = draw.realize(source.width(), source.height());

save_image(drawTest, path);

Example of the drawing:

Source:

Template:

Result:

ASKER'S EDIT2:

made it so it draws only the rectangle around the match

draw(x, y) = select((x == x_coordinate(0, 0) && (y >= y_coordinate(0, 0) && y <= y_coordinate(0, 0) + templ.height())) ||
    ((x == x_coordinate(0, 0) + templ.width()) && (y >= y_coordinate(0, 0) && y <= y_coordinate(0, 0) + templ.height())) ||
    (y == y_coordinate(0, 0) && (x >= x_coordinate(0, 0) && x <= x_coordinate(0, 0) + templ.width())) ||
    ((y == y_coordinate(0, 0) + templ.height()) && (x >= x_coordinate(0, 0) && x <= x_coordinate(0, 0) + templ.width())), 0.0f, limit(x, y));

draw.tile(x, y, x_outer, y_outer, x_inner, y_inner, 64, 64).fuse(x_outer, y_outer, tile_index).vectorize(x_inner).unroll(y_inner).parallel(tile_index);

Result:

edited Aug 24 '16 at 21:13

answered Aug 24 '16 at 17:48

Khouri Giordano

796
3
9

Well yeah if I had any other alternatives I would choose them but at the moment i'm doing this for my thesis along with an OpenCV version and such. Also I'm assuming I have to add the argmin as an update after the template matching is done? Plus I'm wondering how to transfer the coordinates of the pixel to the source image to use it to draw the rectangle which I already have an idea on how to do it. Thanks for answering – Rok Aug 24 '16 at 18:11
Also the RDom I'm using for argmin is basically the entire result picture of template matching – Rok Aug 24 '16 at 18:12
I will try your suggestion and report back though :) – Rok Aug 24 '16 at 18:12
I managed to make a really basic drawing which i'll add to my question. Any tips or just an opinion will be appreciated :) – Rok Aug 24 '16 at 18:54
Seems like I edited the example of it in your answer by mistake – Rok Aug 24 '16 at 18:59
Eh, it's fine. I just noted that you added the edit. – Khouri Giordano Aug 24 '16 at 19:02
You can schedule the `argmin` like `best.update(0) .vectorize(Halide::_0) .parallel(Halide::_1)`. Something like that. – Khouri Giordano Aug 24 '16 at 19:06
Yeah I didn't even come to the part about scheduling but thanks in advance :) – Rok Aug 24 '16 at 19:23
Sadly the schedule doesn't work The reported error is: `In schedule for best, could not find dimension _0 to mark as vectorized in vars for function Vars: __outermost` – Rok Aug 24 '16 at 19:35
1

Define `Func score; score(x,y) = sum(...);` Use `x` and `y` instead of `searchDom.x` and `searchDom.y` in there. Now you can schedule `score` as you did before but add `.compute_root()` at the end. Using any of the inline reductions creates a hidden `Func` that then cannot be scheduled. – Khouri Giordano Aug 24 '16 at 21:11
Wouldn't defining the score as a Func make it not possible to be used in the argmin function since the function needs an Expr? – Rok Aug 25 '16 at 04:12
Let us [continue this discussion in chat](http://chat.stackoverflow.com/rooms/121807/discussion-between-rok-and-khouri-giordano). – Rok Aug 25 '16 at 07:17

Showing result of template matching

1 Answers1