Algorithms: How to highlight image difference using rectangles?

Question

I need to compare two images and create rectangles of the difference. I can build a difference matrix like this:

0  0  0  0  0  0  0  0 
0  0  0  0  0  1  1  1 
0  0  0  1  0  0  0  0 
0  0  0  0  0  0  0  0 
0  0  0  0  1  0  0  0 
0  0  0  0  0  1  0  0 
0  0  0  0  1  1  1  0 
0  0  0  0  1  1  0  0

where 1 is diff pixel. I need to find the way to create rectangles for areas of image diff. In my example, I have three areas to highlight.

# # #
# 1 #
# # #

#  #  #  # 
#  1  1  1 
#  #  #  #

#  #  #  #  #
#  1  0  0  # 
#  0  1  0  # 
#  1  1  1  # 
#  1  1  0  #

So I'm looking for an algorithm to do that in a convenient way.

Find the connected components (e.g. using repeated DFS), and for each just determine its leftmost, rightmost, topmost and bottommost points. — j_random_hacker, Aug 10 '16 at 10:13
The topic is interesting. Could please describe it in a more specific way? [ask] — xenteros, Aug 10 '16 at 10:14
@Finkelson: You don't need to explicitly build a graph -- you can use the graph implicity defined by the grid, where each 1-cell is linked by an edge to whichever of its 4 (or perhaps 8) neighbours are also 1-cells. — j_random_hacker, Aug 10 '16 at 10:17
Just to be absolutely clear: on the output you want a collection of disjoint rectangles (i.e. non-intersecting) describing the regions where the images differ? — blazs, Aug 10 '16 at 10:38
I like the problem a lot. There's **many** solutions. An interesting variation would be to count the number of possible solutions; or, even better, count the number of solutions that describe the region of interest with the minimal number of rectangles possible (in general there could be many such solutions). — blazs, Aug 10 '16 at 10:40
@Finkelson: No, the graph would be undirected. Google "flood fill algorithm" for more help. — j_random_hacker, Aug 10 '16 at 11:02
This is a project (C++) that implements one of the possible approaches: https://github.com/vahancho/nkar — vahancho, Apr 12 '18 at 12:37

blazs · Answer 1 · 2016-08-10T11:03:01.913

I'm assuming that the problem is as follows. Given a 0/1-matrix, cover the regions containing 1s with disjoint rectnagles (i.e. the rectangles must no intersect). In particular, non-rectangular shapes---for example, an L-shaped domino---are disallowed.

Here's an idea for an algorithm:

start at the origin, at index (0,0), and expand out until the expanded region contains a single rectangle of 1s that you cannot enlarge by moving to adjacent cells in either direction.
add that rectangle to the collection, and remove the processed region;
recurse on the remaining cells.

The running time should be linear in the number of cells; however, depending on whether there are additional specifications on the type of the output, you may need to change the first step.

I like the problem a lot. Notice that many different solutions may exist for a problem instance. A natural variation would be to require a cover comprised of as few rectangles as possible (i.e. a minimal cover); in this case, too, there may exist many different solutions. (The counting version of the problem looks interesting from the complexity-theoretic viewpoint.)

Arnauld · Answer 2 · 2016-08-10T11:18:46.627

Below is some JS demonstration code to find the rectangles, starting each time at the next remaining non-empty cell and exploring all paths in a recursive way. Cells are cleared as they are visited.

This is pretty close to what the the 'fill' tool is doing in MS Paint and the like. More precisely, this is a flood fill algorithm, as mentioned by j-random-hacker in the comments.

This code will find the inner bounds of the rectangles. It'd need to be slightly updated if you want the outer bounds instead.

var W = 8, H = 8;
var matrix = [
//  0  1  2  3  4  5  6  7
  [ 0, 0, 0, 0, 0, 0, 0, 0 ], // 0
  [ 0, 0, 0, 0, 0, 1, 1, 1 ], // 1
  [ 0, 0, 0, 1, 0, 0, 0, 0 ], // 2
  [ 0, 0, 0, 0, 0, 0, 0, 0 ], // 3
  [ 0, 0, 0, 0, 1, 0, 0, 0 ], // 4
  [ 0, 0, 0, 0, 0, 1, 0, 0 ], // 5
  [ 0, 0, 0, 0, 1, 1, 1, 0 ], // 6
  [ 0, 0, 0, 0, 1, 1, 0, 0 ]  // 7
];
var dir = [
  [ -1, -1 ], [ 0, -1 ], [ 1, -1 ], [ 1, 0 ], [ 1, 1 ], [ 0, 1 ], [ -1, 1 ], [ -1, 0 ]
];

var x, y, rect;

for(y = 0; y < H; y++) {
  for(x = 0; x < W; x++) {
    if(diffAt(x, y)) {
      rect = { x0:x, y0:y, x1:x, y1:y };
      recurse(x, y, rect);
      console.log(
        'Found rectangle: ' +
        '(' + rect.x0 + ',' + rect.y0 + ') -> ' +
        '(' + rect.x1 + ',' + rect.y1 + ')'
      );
    }
  }
}

function recurse(x, y, rect) {
  rect.x0 = Math.min(rect.x0, x);
  rect.y0 = Math.min(rect.y0, y);
  rect.x1 = Math.max(rect.x1, x);
  rect.y1 = Math.max(rect.y1, y);
  matrix[y][x] = 0;

  for(var d = 0; d < 8; d++) {
    if(diffAt(x + dir[d][0], y + dir[d][1])) {
      recurse(x + dir[d][0], y + dir[d][1], rect);
    }
  }
}

function diffAt(x, y) {
  return x < 0 || x >= W || y < 0 || y >= H ? 0 : matrix[y][x];
}

score 1 · Accepted Answer · answered Aug 10 '16 at 11:16

You could do a two-step algorithm: First, you find 8-connected components in your image, then you calculate the bounding box for each of the component.

This approach may lead to overlapping rectangles (imagine two nearby "L"-shapes), which you could solve by either merging the overlapping rectangles or by zeroing out non-connected components from each rectangle (so that you can sum all the rectangles and reconstruct the difference image appropriately).

If you go with the second choice, you can get the rectangles in Matlab as follows:

%# each element of the struct array rectangles contains a field
%# .Image, which is the rectangle, and 
%# .BoundingBox, which is the coordinates of the rectangle.    
rectangles = regionprops(differenceImage,'Image','BoundingBox');

Algorithms: How to highlight image difference using rectangles?

3 Answers3