How to discard non-rectangular closed regions in image in MATLAB?

Question

I want to extract the rectangular-like shapes (some may have triangular extensions on one side) with an image like this;

What I have done in MATLAB is;

BW=imread('Capture2.JPG');
BW=~im2bw(BW);

SE = strel('rectangle',[1 6]);
BW = ~imclose(BW,SE);
BW2 = imfill(~BW,'holes');

figure
imshow(~BW)

figure
imshow(BW2);

s = regionprops(BW,'BoundingBox','Area','PixelIdxList');
s = s(2:end);
[lab, numberOfClosedRegions] = bwlabel(BW);


figure
imshow(BW)
for i=1:numel(s)
    rec = s(i);
    ratio = rec.BoundingBox(4)/rec.BoundingBox(3);%height/width

%     ratio > 0.3 && ratio < 51.6 && rec.Area > 1100 && rec.Area < 22500
%     if ratio > 0.16

       rectangle('Position',s(i).BoundingBox,'EdgeColor','r','LineWidth',2);
       text(rec.BoundingBox(1),rec.BoundingBox(2),num2str(i),'fontsize',16);
%     end
end

What I have come up is;

As it is seen, there are regions find as part of texts, shape inside of a block(index = 3) and non-block region(index = 11). I need to discard the inside regions and non-block areas.

The other issue is since regions are defined by white areas I need to get the black borders of the blocks so that I can capture the block itself, not the inner white region. How can I solve these issues?

I both tried inverting the image and using the methods but no success.

EDIT: Code improvement & additional image

One of the images can be like this including non-rectangular but object of interest shapes (leftmost).

Another issue if image not as good as it should be some, line considered as open especially diagonal and 1px wide ones which causes regionprops misses them.

Code improvement;

close all;

image=imread('Capture1.JPG');
BW = image;
BW = ~im2bw(BW);


SE = strel('rectangle',[3 6]);
BW = ~imclose(BW,SE); % closes some caps to be closed region

r = regionprops(BW,'PixelIdxList'); 
BW(r(1).PixelIdxList) = 0; %removes outermost white space allowing to connection lines disapear

se = strel('rectangle',[6 1]);
BW = imclose(BW,se);% closes some caps to be closed region
BW = imfill(BW,'holes');

s = regionprops(BW,{'Area', 'ConvexArea', 'BoundingBox','ConvexHull','PixelIdxList'});

%mostly the area and convex area are similar but if convex area is much greater than the area itself it is a complex shape like concave intermediate sections then remove
noidx = [];
for i=1:numel(s)
    rec = s(i);
    if rec.Area*1.5 < rec.ConvexArea 
        BW(rec.PixelIdxList) = 0;
        noidx(end+1)=i;
    end
end

s(noidx)=[];

%no condition for remaining regions figure imshow(BW)

for i=1:numel(s)
    rec = s(i);
    ratio = rec.BoundingBox(4)/rec.BoundingBox(3);%height/width    
%     ratio > 0.3 && ratio < 51.6 && rec.Area > 1100 && rec.Area < 22500
%     if ratio > 0.16
        rectangle('Position',s(i).BoundingBox,'EdgeColor','r','LineWidth',2);
        text(rec.BoundingBox(1),rec.BoundingBox(2),num2str(i),'fontsize',16,'color','red');
%     end
end

Result is;

Advantage is all the remaining regions are region if interest no exception and no condition for area constraint etc. because image size can be different thus the area.

But even this doesn't work on second image. Because of the text below the blocks (which is always the case -> first image was cleared to be uploaded) and the diagonal tips of the leftmost blocks considered open lines.

Is it possible that the rectangles could have thicker lines than the noise? — Anton Savelyev, Jul 26 '17 at 15:03
Don't think. This image is not my creation. I need to work on this. So it is what it is. :/ — freezer, Jul 26 '17 at 15:04
A partial solution is to iteratively check if the center is inside another box. — m7913d, Jul 26 '17 at 15:52
And you can also extend your rectangle by 1 pixel, and then check how many black pixel intersect your rectangle. If nbr_of_black_pixel < pixel_in_perimeter => delete the rectangle. — obchardon, Jul 26 '17 at 16:52
@obchardon: couldn't get what is it used for. What nbr_of_black_pixel < pixel_in_perimeter this says to me? — freezer, Jul 26 '17 at 17:03

score 1 · Answer 1 · answered Jul 26 '17 at 21:22

By adding two conditions I got some good results:

The rectangle need to be fully closed
The area need to be bigger than x pixels (1100 in this case)

In order to check if the rectangle is closed or not, I created an index for each polygon. Those index have the same shape as the rectangles. So if sum(~BW(index)) == sum(index(:)) it mean that the polygon is closed.

The updated code:

warning off

BW=imread('test.jpg');
BW=~im2bw(BW);

SE = strel('rectangle',[1 6]);
BW = ~imclose(BW,SE);
BW2 = imfill(~BW,'holes');


s = regionprops(BW,'BoundingBox','Area','PixelIdxList');
s = s(2:end);
[lab, numberOfClosedRegions] = bwlabel(BW);


figure
imshow(imread('test.jpg'))
inc = 1;
for i=1:numel(s)
    rec = s(i);
    s(i).BoundingBox = floor(s(i).BoundingBox + [-1,-1,2,2]);

    %Creation of the index
    clear ind
    ind = zeros(size(BW));
    y = s(i).BoundingBox(1);
    x = s(i).BoundingBox(2);
    h = s(i).BoundingBox(3);
    w = s(i).BoundingBox(4);
    ind(x:x+w,[y,y+h]) = 1;
    ind([x,x+w],y:y+h) = 1;
    ind = logical(ind);

    if sum(~BW(ind)) == sum(ind(:)) && rec.Area > 1100
       rectangle('Position',s(i).BoundingBox,'EdgeColor','r','LineWidth',1);
       text(rec.BoundingBox(1),rec.BoundingBox(2),num2str(inc),'fontsize',16);
       inc = inc + 1;
    end
end

RESULT

This is a valuable answer but does not solve my problem. I will edit my question accordingly. — freezer, Jul 26 '17 at 21:59

score 0 · Answer 2 · answered Jul 26 '17 at 16:37

0

How can you discard non-rectangular regions? Well I'm sure you can come up with some mathematical properties which are pretty unique to rectangles.

The area is the product of width and heigth.
The perimeter is the twice the sum of width and height.
It obviously has 4 rectangular corners.

I guess the first property would be sufficient and that you can come up with more rules if you need to.

You can get rid of unwanted rectangles and other small stuff with a minimum size constraint or check if they are enclosed by a rectangle.

This should be pretty straight forward.

answered Jul 26 '17 at 16:37

Piglet

27,501
3
20
43

You pointed the very obvious properties which are definitely correct. They can be used. Still don't know how to discard region (index = 11) and some shapes (didn't shown here) has non-rectangular but convex shapes like triangles at the tip of the rectangle like bookmark shape. – freezer Jul 26 '17 at 17:00
@freezer I don't understand what you want. The shapes you describe are no rectangles. Hence they cannot fullfill the constraints I mentioned. Area = width * height alone is sufficient to get rid of them. – Piglet Jul 27 '17 at 06:43
You are right I may have written the title of the question wrong. But, I mentioned this situation more than once. There maybe non-rectangular shapes which are also interested. The parts to be discarded are the area between the connection lines. There maybe lots of variations but I guess they are non-convex areas. – freezer Jul 27 '17 at 09:41
@freezer you have two options. a) you identify all known shapes of interest or b) you remove unwanted shapes. I guess you'll have to classify the shapes anway at some point so why not classify them right now and skip anything else. these objects are drawn by a computer so their geometry follows certain rules which you can use for your search. if the area between your objects is the only concave thing you can of course use this fact to get rid of them. – Piglet Jul 27 '17 at 10:02
How can I identify objects and their position. My aim is to give the type position and eventually the name of the object so that I can recreate the model in the software from image. – freezer Jul 27 '17 at 10:08
@freezer that would too much to discuss here. do some research on shape analysis, shape descriptors, template matching, contour matching... I don't know how many symbols you have to deal with. but that rectangle with the triangle on its right side also has a specific area that is a function of its width and height, so if you don't have too many symbols you could get around with a few sets of simple rules. – Piglet Jul 27 '17 at 10:21
You are right. I can analyze aspect ratios, areas etc. But one of my concern is all the images will be analyzed has not the same size thus same icon have different area for different scales. I looked for traffic sign recognition things a bit but there are many sets of images to train the network. – freezer Jul 27 '17 at 10:23
@freezer but as long as you don't change the aspect ratio size does not matter. your constraints have to use scale-invariant features – Piglet Jul 27 '17 at 10:33
Yes but I guess I need to use more advanced methods to find the type of object. I cannot be sure if the block wasn't scaled on just one axis to widen. – freezer Jul 27 '17 at 10:36
@freezer if you widen a rectangle it still remains a rectangle... same for all other features. they are drawn applying mathematical rules and they don't change. and even if only the image was changed you could still compensate that if you have any feature of known original aspect ratio. and there are still many features that are invariant to changes in aspect ratio as well – Piglet Jul 27 '17 at 10:49
thanks for your all comments. I will work on the things you said. Let's see what will happen. – freezer Jul 27 '17 at 11:08

How to discard non-rectangular closed regions in image in MATLAB?

2 Answers2