How to compute moments to find centroid from largest blob using contourArea OpenCV

Question

I want to detect yellow colour objects and plot a centroid position on the largest detected yellow object.

I perform my steps in this order:

converted the input rgbaframe to hsv using cvtColor() method
perform color segmentation in HSV using inRange() method, bound it only to yellow color ranges & returns a binary threshold mask.
I perform morphology operation (specifically MORPH_CLOSE) to perform dilation then erosion on the mask to remove any noise.
I perform a gaussian blur to smooth the mask.
I apply canny algorithm to perform edge detection to make edges more obvious to prepare for Contour detection in the next step. (i'm starting to wonder if this step is of beneficial use at all?)
I apply findContour() algorithm to find the Contours in the image as well as find the hierarchy.
HERE I intend to use feature2d.FeatureDetection(SIMPLEBLOB)& pass in the blob area for detection as Params, however there seems to be no implementation supporting for Android, hence I had to work around the limitation and find largest blob using Imgproc.contourArea().

is there a way to do so?

I pass in the contours obtained previously from the findContours() method as parameter to Imgproc.moments to compute the Centroid Position of the objects detected.

HOWEVER, I would like to bring to everyone's attention that this current implementation will compute all centroids in EACH contour (yellow objects) detected. *PLS SEE/REFER to pic 1, 2 to see what is output onto the Frame back to the user.

What I would like to achieve is to find a way to use the contour of the largestblob (via largestContourArea) and pass that info on as a parameter into the ImgprocMoments() so that I will ONLY COMPUTE the centroid of that largest Contour(Object) detected, so I should only see 1 centroid Pos plotted on the screen at any particular point in time.

I have tried a few methods such as passing the contour of the Largest object as param into Imgproc.moments() but it didn't work either due to difference in data Type / if it worked, the output is not as desired, w multiple centroid points being plotted within or along the perimeter of the object, rather than 1 single point at the center of the largest contour object.

public Mat onCameraFrame(CameraBridgeViewBase.CvCameraViewFrame inputFrame) {

     InputFrame = inputFrame.rgba();

     Core.transpose(InputFrame,mat1); //transpose mat1(src) to mat2(dst), sorta like a Clone!
     Imgproc.resize(mat1,mat2,InputFrame.size(),0,0,0);    // params:(Mat src, Mat dst, Size dsize, fx, fy, interpolation)   Extract the dimensions of the new Screen Orientation, obtain the new orientation's surface width & height.  Try to resize to fit to screen.
     Core.flip(mat2,InputFrame,-1);   // mat3 now get updated, no longer is the Origi inputFrame.rgba BUT RATHER the transposed, resized, flipped version of inputFrame.rgba().

     int rowWidth = InputFrame.rows();
     int colWidth = InputFrame.cols();

     Imgproc.cvtColor(InputFrame,InputFrame,Imgproc.COLOR_RGBA2RGB);
     Imgproc.cvtColor(InputFrame,InputFrame,Imgproc.COLOR_RGB2HSV);


     Lower_Yellow = new Scalar(21,150,150);    //HSV color scale  H to adjust color, S to control color variation, V is indicator of amt of light required to be shine on object to be seen.
     Upper_Yellow = new Scalar(31,255,360);    //HSV color scale


     Core.inRange(InputFrame,Lower_Yellow, Upper_Yellow, maskForYellow);


     final Size kernelSize = new Size(5, 5);  //must be odd num size & greater than 1.
     final Point anchor = new Point(-1, -1);   //default (-1,-1) means that the anchor is at the center of the structuring element.
     final int iterations = 1;   //number of times dilation is applied.  https://docs.opencv.org/3.4/d4/d76/tutorial_js_morphological_ops.html

     Mat kernel = Imgproc.getStructuringElement(Imgproc.MORPH_ELLIPSE, kernelSize);

     Imgproc.morphologyEx(maskForYellow, yellowMaskMorphed, Imgproc.MORPH_CLOSE, kernel, anchor, iterations);   //dilate first to remove then erode.  White regions becomes more pronounced, erode away black regions



     Mat mIntermediateMat = new Mat();
     Imgproc.GaussianBlur(yellowMaskMorphed,mIntermediateMat,new Size(9,9),0,0);   //better result than kernel size (3,3, maybe cos reference area wider, bigger, can decide better whether inrange / out of range.
     Imgproc.Canny(mIntermediateMat, mIntermediateMat, 5, 120);   //try adjust threshold   //https://stackoverflow.com/questions/25125670/best-value-for-threshold-in-canny

     List<MatOfPoint> contours = new ArrayList<>();
     Mat hierarchy = new Mat();
     Imgproc.findContours(mIntermediateMat, contours, hierarchy, Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_SIMPLE, new Point(0, 0));   

     byte[] arr = new byte[100];
     //List<double>hierarchyHolder = new ArrayList<>();
     int cols = hierarchy.cols();
     int rows = hierarchy.rows();
     for (int i=0; i < rows; i++) {
         for (int j = 0; j < cols; j++) {
            //hierarchyHolder.add(hierarchy.get(i,j));
            //hierarchy.get(i,j) is a double[] type, not byte.
            Log.d("hierarchy"," " + hierarchy.get(i,j).toString());   

        }
    }


     double maxArea1 = 0;
     int maxAreaIndex1 = 0;
     //MatOfPoint max_contours = new MatOfPoint();
     Rect r = null;
     ArrayList<Rect> rect_array = new ArrayList<Rect>();

     for(int i=0; i < contours.size(); i++) {
         //if(Imgproc.contourArea(contours.get(i)) > 300) {   //Size of Mat contour @ that particular point in ArrayList of Points.
         double contourArea1 = Imgproc.contourArea(contours.get(i));    
        //Size of Mat contour @ that particular point in ArrayList of Points.
             if (maxArea1 < contourArea1){
                 maxArea1 = contourArea1;
                 maxAreaIndex1 = i;
             }
             //maxArea1 = Imgproc.contourArea(contours.get(i));  //assigned but nvr used
             //max_contours = contours.get(i);
             r = Imgproc.boundingRect(contours.get(maxAreaIndex1));    
             rect_array.add(r);  //will only have 1 r in the array eventually, cos we will only take the one w largestContourArea.
     }


     Imgproc.cvtColor(InputFrame, InputFrame, Imgproc.COLOR_HSV2RGB);


     if (rect_array.size() > 0) {   //if got more than 1 rect found in rect_array, draw them out!

         Iterator<Rect> it2 = rect_array.iterator();    //only got 1 though, this method much faster than drawContour, wont lag. =D
         while (it2.hasNext()) {
             Rect obj = it2.next();
             //if
             Imgproc.rectangle(InputFrame, obj.br(), obj.tl(),
                     new Scalar(0, 255, 0), 1);
         }

     }


 //========= Compute CENTROID POS! WHAT WE WANT TO SHOW ON SCREEN EVENTUALLY!====================== 

     List<Moments> mu = new ArrayList<>(contours.size());    //HUMoments
     for (int i = 0; i < contours.size(); i++) {
         mu.add(Imgproc.moments(contours.get(i)));
     }

     List<Point> mc = new ArrayList<>(contours.size());   //the Circle centre Point!
     for (int i = 0; i < contours.size(); i++) {
         //add 1e-5 to avoid division by zero
         mc.add(new Point(mu.get(i).m10 / (mu.get(i).m00 + 1e-5), mu.get(i).m01 / (mu.get(i).m00 + 1e-5)));
     }


     for (int i = 0; i < contours.size(); i++) {
         Scalar color = new Scalar(150, 150, 150);

         Imgproc.circle(InputFrame, mc.get(i), 20, color, -1);   //just to plot the small central point as a dot on the detected ImgObject.
     }

Picture of output when view on CameraFrame:

1:

2:

score 0 · Answer 1 · answered Sep 23 '19 at 11:07

Managed to resolve the issue, instead of looping thru the entire array of Contours for yellow objects detected, and pass each contour as parameter to Imgproc.moments, I now only assign the Contour at the particular index which the LargestContour is detected, so only 1 single contour is being processed now by Imgproc.moments to compute Centroid! The code correction is as shown below

//========= Compute CENTROID POS! WHAT WE WANT TO SHOW ON SCREEN EVENTUALLY!======================
        List<Moments> mu = new ArrayList<>(contours.size());
        mu.add(Imgproc.moments(contours.get(maxAreaContourIndex1)));    //Just adding that 1 Single Largest Contour (largest ContourArea) to arryalist to be computed for MOMENTS to compute CENTROID POS!

        List<Point> mc = new ArrayList<>(contours.size());   //the Circle centre Point!
        //add 1e-5 to avoid division by zero
        mc.add(new Point(mu.get(0).m10 / (mu.get(0).m00 + 1e-5), mu.get(0).m01 / (mu.get(0).m00 + 1e-5)));   //index 0 cos there shld only be 1 contour now, the largest one only!
        //notice that it only adds 1 point, the centroid point. Hence only 1 point in the mc list<Point>, so ltr reference that point w an index 0!

        Scalar color = new Scalar(150, 150, 150);

        Imgproc.circle(InputFrame, mc.get(0), 15, color, -1);   //just to plot the small central point as a dot on the detected ImgObject.

How to compute moments to find centroid from largest blob using contourArea OpenCV

1 Answers1