0

I'm trying to draw a boundary box around the different characters in the image below. However, as some of the characters are joined, I can't have the box drawn. I have tried a few things such as; dilating, eroding and trying different blurs but I can't seem to get them apart. Eroding seems to be the best way to get close but If I continue to keep eroding, the characters can't be recognized.

I'm using the OpenCV library in Java to achieve this.

Mat img = Imgcodecs.imread("test.jpg");

        List<MatOfPoint> contours = new ArrayList<>();
        Mat hierarchy = new Mat();
        Imgproc.cvtColor(img, img, Imgproc.COLOR_BGR2GRAY);
        Imgproc.threshold(img, img, 220, 255, 0);
        Imgproc.erode(img, img, Imgproc.getStructuringElement(Imgproc.MORPH_RECT, new Size(3, 3)));
        Imgproc.findContours(img, contours, hierarchy, Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_NONE);


        MatOfPoint2f approxCurve = new MatOfPoint2f();

        //For each contour found
        for (int i = 0; i < contours.size(); i++) {
            MatOfPoint2f contour2f = new MatOfPoint2f(contours.get(i).toArray());
            double approxDistance = Imgproc.arcLength(contour2f, true) * 0.02;
            Imgproc.approxPolyDP(contour2f, approxCurve, approxDistance, true);

            MatOfPoint points = new MatOfPoint(approxCurve.toArray());

            Rect rect = Imgproc.boundingRect(points);
            Imgproc.rectangle(img, rect.tl(), rect.br(), new Scalar(255, 255, 255), 1, 8, 0);
        }
        System.out.println(img);
        Imgcodecs.imwrite("o.jpg", img);

Before:

https://i.imgur.com/qiTwfnx.png

After:

https://i.imgur.com/ekjbWNs.png

JJIqbal
  • 630
  • 1
  • 8
  • 23
poofit
  • 13
  • 4
  • if the digits share so many pixels your approach isn't going to work. suggesting another approach would require more knowledge about the variations in font, scale, rotation,... – Piglet Apr 04 '19 at 15:25
  • Try to do an erosion with a bigger structuring size, big enough that the numbers are distinguished. I think you are doing 3x3. That is too small, maybe 7x7. Then do dilation with the same size to bring back the original shape (kind of). A combined erosion+dilation is known as _morphological opening_. Something like `Imgproc.erode(img, img, Imgproc.getStructuringElement(Imgproc.MORPH_RECT, new Size(7, 7)));` `Imgproc.dilate(img, img, Imgproc.getStructuringElement(Imgproc.MORPH_RECT, new Size(7, 7)));` – Tetix Apr 04 '19 at 15:35
  • @Piglet Yeah they do share quite a few pixels. I'm using a library to generate these images. The library is called: simplecaptcha ([link]http://simplecaptcha.sourceforge.net/ ) I'm not sure how to check for font, scale etc – poofit Apr 04 '19 at 15:35
  • @Tetix I've just tried it and setting it at 7 on both removes the issue completely. I've adjusted the values and 5 is the best I can go. https://i.imgur.com/gW0UNbs.png – poofit Apr 04 '19 at 15:42
  • Is this the result with 5x5? What happens with 7x7? – Tetix Apr 04 '19 at 15:47
  • This is the result with 7x7: https://i.imgur.com/Vhq7V4f.png – poofit Apr 04 '19 at 15:50
  • 2
    if connected components was anything good to solve captchas, captchas were completely useless. I guess you have to dig deeper ;) it's like, hey I want to pick this lock but I can't get my fingers in. can you help me? – Piglet Apr 04 '19 at 15:51
  • @Piglet Very well put! I'll keep digging and trying things :) – poofit Apr 04 '19 at 16:25

1 Answers1

0

One possible way (you could try) will work is if you know the count of the characters, e.g: 5, then you can first bound a rectangle around the segmented contours then, divided the width by 5, and then draw the rectangles with black color on the image, and then the image will be successfully segmented.

Follow the steps below:

cv::namedWindow("result", cv::WINDOW_FREERATIO);
cv::Mat img = cv::imread(R"(D:\A49d6.png)");

// to gray
cv::Mat gray;
cv::cvtColor(img, gray, cv::COLOR_BGR2GRAY);
// remove unvisible noise
cv::threshold(gray, gray, 20, 255, cv::THRESH_BINARY);

// this is to make sure to get the correct bounding rect
cv::morphologyEx(gray, gray, cv::MORPH_CLOSE, cv::getStructuringElement(cv::MORPH_CROSS, cv::Size(5, 1)));

std::vector<std::vector<cv::Point> > contours;
cv::findContours(gray, contours, cv::RETR_EXTERNAL, cv::CHAIN_APPROX_SIMPLE);

// because we know that there is only one contour, so get the first posittion (0 index) from the list
cv::Rect rect = cv::boundingRect(contours[0]);

// calculate the width for each character
int width = int(round(rect.width/5.0));
for(int i(0); i<5; i++) {
    cv::Rect r(rect.x+(width*i), rect.y, width, rect.height);
    cv::rectangle(img, r, cv::Scalar(0));
}

cv::imshow("result", img);
cv::waitKey();

And this is the result:

enter image description here

Now the image is clear enough for recognition.

Bahramdun Adil
  • 5,907
  • 7
  • 35
  • 68