2

I am new to Open CV and am working on a ALPR solution based on violation images from multiple cameras. In most cases I am able to extract the text from the plate and run OCR however if the image brightness/contrast is out this poses a problem.

I've transferred my code to a console app and pasted below.

class Program
{
    static void Main(string[] args)
    {
       
            var path = @$"path to image";
            var img = LoadImage(path);

            var resized = Resize(img);

            var gray = Gray(resized);

            var otsu = SimpleThresh(gray, 0, 43, ThresholdType.Otsu);

            var binary = SimpleThresh(otsu, 0, 255, ThresholdType.Binary);
            DetectText(inverse);
            Show(binary, "binary");
        }
    }

    static Mat LoadImage(string path)
    {
        return CvInvoke.Imread(path);
    }

    static Mat SimpleThresh(Mat img, double low, double max, ThresholdType type)
    {
        var th = new Mat();
        CvInvoke.Threshold(img, th, low, max, type);
        return th;
    }

    static Mat Brighten(Mat img, double alpha=1.95, double beta=0)
    {

        Mat bright = new Mat();
        var mean = CvInvoke.Mean(img);
        CvInvoke.ConvertScaleAbs(img, bright, alpha, beta);
        return bright;
    }

    static Mat Resize(Mat orginalImage)
    {
        var newHeight = (int)(orginalImage.Height + (orginalImage.Height * 2));
        var newWidth = (int)(orginalImage.Width + (orginalImage.Width * 2));
        Mat img = new Mat();
        CvInvoke.Resize(orginalImage, img, new Size(newWidth, newHeight), 0, 0, Inter.Cubic);
        return img;
    }

    static Mat Gray(Mat img)
    {

        Mat gray = new Mat();
        CvInvoke.CvtColor(img, gray, ColorConversion.Bgr2Gray);
        return gray;
    }      

    static void Show(Mat img, string name)
    {
        CvInvoke.Imshow(name, img);
        CvInvoke.WaitKey();
    }

    
    static string DetectText(Mat img, string sourceFilePath )
    {
        var path = @"C:\tesseract";

        using (var engine = new TesseractEngine(path, "eng"))
        {

            //engine.SetVariable("user_defined_dpi", "700");
            engine.SetVariable("tessedit_char_whitelist", "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789");
            engine.DefaultPageSegMode = PageSegMode.SingleChar;
            using ( var img1 = Pix.LoadFromFile(sourceFilePath))
            {
                using (var page = engine.Process(img1))
                {
                    var text = page.GetText();
                    Console.WriteLine();
                    Console.WriteLine("---Image Text---");
                    Console.WriteLine();
                    Console.WriteLine(text);
                }
            }
        }

        var Ocr = new IronTesseract(); // nothing to configure
        Ocr.Language = OcrLanguage.English;
        Ocr.Configuration.TesseractVersion = TesseractVersion.Tesseract5;
        Ocr.Configuration.PageSegmentationMode = TesseractPageSegmentationMode.SingleChar;
        Ocr.Configuration.WhiteListCharacters = "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789";
        using (var Input = new OcrInput())
        {
            Input.AddImage(Emgu.CV.BitmapExtension.ToBitmap(img));
            var Result = Ocr.Read(Input);
            Console.WriteLine(Result.Text);
            return Result.Text;
        }
    }

}

This code works well for in most cases, I get a nice binary and then loop through contours and cleanup before sending through to OCR engine. However if the brightness and contrast of the image is low then the binary thresholding does not work for me.

Example image where the thresholding fails

enter image description here

Below I have a screenshots of outputs from each stage

enter image description here

Example of success

enter image description here

screenshots of success outputs

enter image description here

If I brighten the image by using the Brighten() method I have posted above this resolves the problem however I have no idea which images need adjusting because I am working with mass amount of data.

Is there any way I can use c#/emgu.cv auto detect cases that require brightness/contract adjustments and set the brightness/contrast?

Or if there is an alternative route to acheive ALPR please let me know, as mentioned I am new to this.

Christoph Rackwitz
  • 11,317
  • 4
  • 27
  • 36
Asaad
  • 141
  • 1
  • 5
  • so you're saying otsu fails? show your outputs too. do you really run this on a picture with a red border? why do you even add that red border? it's ruining everything. – Christoph Rackwitz Sep 24 '22 at 12:28
  • @ChristophRackwitz apologies, you can ignore the red border, thats just a bounding box I use when displaying the results - I will edit the ansewer with the outputs – Asaad Sep 24 '22 at 13:02
  • I have removed the redboxes and also displayed the outputs - see edited post – Asaad Sep 24 '22 at 13:16
  • 1
    ah. yes, otsu can fail and it fails here. stretched histogram: https://i.stack.imgur.com/0G2rd.png it simply won't find the black lettering (small mode). it's going for the two big modes, which are the car (white) and the sign (gray/yellow). -- for that specific image, I'd choose an approach using normalize(MINMAX) and then a fixed but low threshold (say 64) -> https://i.stack.imgur.com/b1sFh.png I'm wondering if I should write up an answer... because that doesn't work on your other picture, the one with red parts – Christoph Rackwitz Sep 24 '22 at 15:07
  • 1
    this really shouldn't be solved by messing with the image explicitly. a proper neural network can decode these pictures as-is. tesseract simply is that bad. it's decades old tech, nobody should be using it. – Christoph Rackwitz Sep 24 '22 at 15:13
  • Thanks for your response - I'm new to this and wasn't sure if I'm taking the correct approach - your OTSU explanation makes sense. I can probably crop this image further so it only includes the plate. I'm using ML.net object detection and in most cases I get a close up of the plate which is probably why it works. Will look into your neural network suggestion - if you have any pointers let me know. Would love to move away from tesseract – Asaad Sep 24 '22 at 16:01

0 Answers0