Find enclosing rectangle of image object using second moments

Question

My understanding is the second central moment should give me an object's variance. Which I should be able to use as a measure of distribution of pixels from the center of the object, and so I would think I could find the size of the enclosing rectangle of an object using the second moments. Say in Malab I create an image with a white rectangle of dimensions 100x200 and find the center of mass using the first moments, then I calculate the variance

im = zeros(501,400);
im(200:300,100:300) = ones(101,201);
%Sum the moments in a for loop for clarity
for v = 1:rows  
  for u = 1:cols
    val = im(v,u);
    %zeroth moment is just the total pixel count
    m00 = m00 + val;        
    %First moments. pixel value times position is the m10 and m01
    m01 = m01 + (val * v);
    m10 = m10 + (val * u);
   end
end
%centers of mass
uc = m10/m00;
vc = m01/m00;
%Now I find the second central moments wrt centroid 
for v = 1:rows  
  for u = 1:cols
    val = im(v,u);
    u02 = u02 + (v - vc)^2 * val;
    u20 = u20 + (u - uc)^2 * val;
  end
end
hold on;
plot(uc,vc, 'r+');
text(uc + 5,vc + 5,strcat('X:',num2str(uc)),'Color','green','FontSize',12);
text(uc + 5,vc + 20,strcat('Y:',num2str(vc)),'Color','green','FontSize',12);

My value for u02 is 17255850, the square root of which is 4154. Is there a way I can directly get the approximate size of my rectangle using the variance? Thanks for any help.

EDIT: Experimenting with fitting an ellipse using the covariance matrix and trying to calculate the size using the variance as discussed in the comments and I get the image below. The dimensions of the yellow rectangle were calculated with

  %can we estimate size based on standard deviation?
stdX = sqrt(u20/m00);
stdY = sqrt(u02/m00);
rectangle('Position',[uc-2*stdX vc-2*stdY 4*stdX 4*stdY],'EdgeColor','yellow');

No, not in general. You can only get the size of the “best fit” ellipse. It depends in the shape of the object how that ellipse translates to a minimal enclosing box. If you know your object is a solid rectangle, you can calculate its size from the second order moments. — Cris Luengo, Feb 22 '21 at 13:46
Thanks, I'm experimenting with the best fit ellipse using the covariance matrix and that works well., But was trying to calc the size from just the 2nd order moments for simple shapes such as circles and rectangles, just trying to understand. To get the size from the 2nd order moments would I take my above code and add sqrt(u20/m00) wouldn't that give me the standard deviation in the x-direction? Then estimate an enclosing rectangle as uc +/- 2 * stdev? I guess it depends on if the pixels follow a normal distribution? — mash, Feb 22 '21 at 14:23
The rectangle is not +/- 2 stddev. The variance is 1/12 width^2, so width=sqrt(12*u20/m00) or something like that. Let me look this up and post an actual answer. — Cris Luengo, Feb 22 '21 at 14:29

score 1 · Accepted Answer · answered Feb 22 '21 at 16:28

1

After you've computed your normalized second order central moments, you get a matrix like this:

| u20/m00  u11/m00 |
| u11/m00  u02/m00 |

The two eigenvalues of this matrix provide a rotationally-invariant measure of the dimensions of the object.

%Normalized second order central moment tensor
mu = [u02/m00, u11/m00; u11/m00, u20/m00];
%The eigenvalues relate to the size
[V,D] = eig(mu);
l1 = D(1,1);
l2 = D(2,2);
%The eigenvectors relate to the orientation
phi = atan2(V(1,2),V(1,1));

Given the eigenvalues, we can determine the radii of the ellipse with the same normalized second order central moments:

a = 2*sqrt(l1);
b = 2*sqrt(l2);

Note that this is true only for a solid ellipse.

One can consider this the best fit ellipse, for some definition of "best".

We cannot determine the size of an arbitrary shape for these values, but if we know that our shape is a rectangle, then we can determine its sizes by knowing that the moments of inertia of a rectangle in 2D are (see Wikipedia, compare moments of 3D cuboid):

| 1/12 m h^2       0     |
|      0      1/12 m w^2 |

Where m is the mass, which in our case is 1 since we already normalized the moments.

Thus,

w = sqrt(12*l1);
h = sqrt(12*l2);

which gives us w = 100.9950 and h = 200.9975 for the case in the OP.

answered Feb 22 '21 at 16:28

Cris Luengo

55,762
10
62
120

Very helpful, Thank you! If we did have an arbitrary shape, and the goal is to, say, just draw an enclosing rectangle around an object. Would the best approach be to use the radii of the best fit ellipse? Thanks again for the great answer. – mash Feb 22 '21 at 18:38
1

@mash: The simplest way to find the bounding rectangle is to find the minimum and maximum x and y coordinates. That is a lot cheaper than computing moments too. Iterate over the image, for each object pixel you record `minx = min(minx,x); maxx = max(maxx,x);` and the same for the y coordinate of that pixel. That leads to two corners that define the rectangle: `(minx,miny), (maxx,maxy)`. – Cris Luengo Feb 22 '21 at 19:53
I'm really just learning and experimenting with the higher order moments. One last question, if I knew my shape were say a circle, then according to the 3D cuboid link you posted where for circle a Ix = 1/2 mr^2 could I find the radius with something like r = sqrt(2 * L1)? And yes you're right about iterating and finding min(x) etc... being much easier and is what I usually do. Thanks again – mash Feb 22 '21 at 20:05
1

@mash: A circle is not solid, you want to look at the disk. But the moments of the 2D shape in 3D are not identical to the moments in a 2D world. A circle is just an ellipse with `a==b`, so you get `r = 2*sqrt(l1)` as I indicated in the answer (the diameter is given by `sqrt(16*l1)`). – Cris Luengo Feb 22 '21 at 20:38

Find enclosing rectangle of image object using second moments

1 Answers1