0

I am currently doing a project of spotting fractured or bad products from a stream of product line. I am trying to use Opencv to extract the edges by Canny. Below are the edges attained from a bad product and a sample product respectively.

BAD

Bad

GOOD

Good

The Edges are quite clear but there are many small regions of meaningless spots (noises) so it seems impossible to directly compare these two photos pixel by pixel. I was thinking if I can keep rotating and shifting the bad product and find the best comparison score such as square difference between pixels. However, with these noises, this method seems to be impossible.

Therefore, I am thinking if there is a way to extract the largest connected regions or split those connected regions into a few pictures? Or, if there exists a better comparison method even under these circumstances?

Thanks a lot!!!

Community
  • 1
  • 1
Hiu leong Ng
  • 33
  • 8
  • CAn you upload the original image? – PSchn Sep 10 '16 at 10:12
  • Sorry, i couldnt upload the original images at the moment. Without the original images, will it be too difficult to spot out the products? – Hiu leong Ng Sep 10 '16 at 10:19
  • 1
    Did you blur the image before detecting edges? This should reduce noise artefacts! Then you can detect all contours and filter out small ones. Maybe you can try to take just the external contours and ignore their children. See [here](http://docs.opencv.org/2.4/modules/imgproc/doc/structural_analysis_and_shape_descriptors.html?highlight=findcontours#findcontours) for a detailed explanation and use. – PSchn Sep 10 '16 at 10:49
  • You can try to use [matchShapes](http://docs.opencv.org/2.4/modules/imgproc/doc/structural_analysis_and_shape_descriptors.html#matchshapes) to compare the found contour to your reference. – PSchn Sep 10 '16 at 14:08
  • Thanks for your suggestion!! I have tried to blur first then do the edge detection.It seems like the noise is reduced but there still some noise in the image. Therefore, it may affect the matchShapes comparison. Also, I have tried to find and draw the external contours in color using draw contours function. There are quite many colors in the edge. Does it mean the edge is not connected ? Will it be better for me to connect those related contours first? Is there any function in opencv to connect those contours?Thanks!! – Hiu leong Ng Sep 10 '16 at 16:33
  • Below is the link of good picture after blurred and the drawn contour https://drive.google.com/file/d/0B9_vlz2OS7h9NmdDVTA1akQyV2M/view?usp=sharing – Hiu leong Ng Sep 10 '16 at 16:37
  • Moreover, for the MatchShapes function, which metrics will you recommend? Thanks a lot!! – Hiu leong Ng Sep 10 '16 at 16:37
  • That means that findContours find a lot of contours ;) and yes i think so. Try to tune the canny parameters to get a closed contour. If this is not successful, try to merge them with morphology. or you start using a different approach. its hard to say without your original image and some more information. – PSchn Sep 10 '16 at 18:25
  • Thanks for ur suggestion. Would you mind further elaborate how morphology works? Which function should i use? I think it is hard to tune canny parameters ... What are the different approaches i could use? Sorry that i couldnt upload the original image at the moment. – Hiu leong Ng Sep 11 '16 at 01:39
  • 1
    Getting continous edges is indeed a challenge. It might be that the outline of your part is easier to get by simple thresholding, but you didn't share the original image. –  Sep 12 '16 at 08:53
  • Sorry that I couldnt upload the original image here but I have tried to do some simple thresholding and the result seems not so good. Therefore, I chose not to do thresholding. But, maybe it is just because my thresholding was not correct. So, would you mind elaborating more on the way of trying thresholding ? Thanks!!! – Hiu leong Ng Sep 17 '16 at 16:56

1 Answers1

0

So i try to explain one possible solution here. After you blurred your image and got your edge image you can thicken the found edges with dilation. If you have a set X and a structured element B, dilation can be understood as set of the points x covered by B when the center of B is inside X:

enter image description here

enter image description here

If you want to read more about this topic i recommend you to P.Soille:Morphological Image Analysis. Know your edges are thicker and connected. After findContours you pick the largest contour (you should do some more sophisticated tests). Here are the results i got:

Edges after dilation: enter image description here

int dilation_type = MORPH_RECT;//MORPH_CROSS,MORPH_ELLISPE
int dilation_size = 2;
Mat element = getStructuringElement( dilation_type,Size( 2*dilation_size + 1,2*dilation_size+1 ),Point( dilation_size, dilation_size ));
dilate( src, erosion_dst, element );

Found contours: enter image description here

Bounding rectangle of biggest contour: enter image description here

PSchn
  • 718
  • 4
  • 14
  • Thank for your suggestion!!! With your help and code, I can currently find out the largest contour of the good sample picture with comparing the contour area function in opencv. However, it seems work only when the edge is perfect such as the good sample. For the bad sample, even after dilation, the outer edge is still disconnected. Could I just increase the dilation_size to try to connect the edges? Also, is it correct for me to compare contour just by Contour area? I am curious how opencv calculated the area for contour which is not closed. It seems like the area calculated is not zero. – Hiu leong Ng Sep 12 '16 at 06:21
  • The largest contour found in the good and bad picture is following: https://drive.google.com/open?id=0B9_vlz2OS7h9NmdDVTA1akQyV2M – Hiu leong Ng Sep 12 '16 at 06:23
  • mhmm yes you can try to dilate several times or try to use a closing. But i think this is not the best solution. The next image can be worst and your algortihm fails, or you merge contours you dont want to be together,..... You can try to combine nearby contours by their distance, but that leads to false merging too i guess. Did you try an segmentation by thresholding (global or adaptive)? Do you have a color image? If so, can you segment your object in HSV? – PSchn Sep 12 '16 at 09:26
  • Thanks for your suggestion. In fact, the edge detected in my program was found by first splitting the colored source image into HSV and then perform canny edge detection on H portion. However, I didnt do any segmentation by thresholding in my coding. Could you mind further explaining how I should perform the thresholding? For example, before canny edge detection ? Which thresholding should I use? What is the purpose of thresholding? Really thank you for your suggestion and help!!!! – Hiu leong Ng Sep 12 '16 at 09:33
  • Thats really hart to say without the original image. First i would recommend you to download [ImageJ](https://imagej.nih.gov/ij/). Its a java baised image-processing tool. Here you can perform basic image operations like thresholding and pick the best values from the histogramm. And then you can realize it in opencv. You can perform the thresholding in the HSV-space with opencv's `inRange`. Or you can convert your image to grayscale, or take one RGB-channel with high contrast and do a thresholding with `threshold` or `adaptiveThreshold`. The result is a binary image, so no canny needed. – PSchn Sep 12 '16 at 09:46
  • Thanks for your help!! Currently, as the bad images are very likely to have unconnected contours, I am planning to compare the contours found at the current stage and see the testing error of it. I am trying to use the MatchShapes function you mentioned before but it seems not too suitable for the objects to be tested because the objects are made of plastic and would not have a fixed size. So, I have found two functions in opencv (HausdorffDistanceExtractor, shapeContextDistanceExtractor) Do you have any ideas about these two methods? Thanks a lot !!!!!! – Hiu leong Ng Sep 16 '16 at 05:02
  • I know Hausdorff-Distance from 3D, it computes the distance from A->B and B->A and takes the maximum. I dont know the other method. I have to ask it again, is there any way you to manipulate the image acquisition to get better contrast between background and your object? It would make your life much easier :D – PSchn Sep 16 '16 at 06:45
  • Thanks for your suggestion. I remembered the result from Hausdorff-Distance were quite strange. That distance from two duplicated image is not zero..... Will it give a closer distance if the images to be compared are similar in shape? Thanks! – Hiu leong Ng Sep 16 '16 at 08:05
  • I was following the shape distance usage for Hausdorff Distance calculation. In the following sample, would you mind explaining me the meaning of using SimpleContour defined inside the code? https://github.com/opencv/opencv/blob/master/samples/cpp/shape_example.cpp – Hiu leong Ng Sep 16 '16 at 08:16
  • I found that the distance calculated after performing simpleContour defined in that example on the image drawn with the largest contour will be different from I just select the largest contour of the image. Do you know the reason behind? Thanks !!!! – Hiu leong Ng Sep 16 '16 at 08:18
  • I have tried to compare the sample image with 19 testing samples. Below is the results (the largest contour found) and the distance calculated by Hausdoff and the three methods (I1, I2, I3) in the matchshape function. This time, the object to be tested should be similar to a circle. But, the result doesnt really align with the shape found (see if they are circle, for example image 9)... Would you mind helping me to have a look on the contours found and suggesting me the way of comparison? Thanks a lot !!!! https://drive.google.com/file/d/0B9_vlz2OS7h9dnV5S2R6bXRjQ1U/view?usp=sharing – Hiu leong Ng Sep 16 '16 at 10:54
  • Moreover, is the matchshape function invariant of translation and rotation? I have heard that fourier descriptor will be invariant of translation and rotation and can be used to compare images as well. Have you used that before? Thanks!!! – Hiu leong Ng Sep 16 '16 at 15:38
  • Hope there are someone to answer these questions. Thanks! – Hiu leong Ng Oct 02 '16 at 11:23