Denoise and filter an image

Question

I am doing a license-plate recognition. I have crop out the plate but it is very blurred. Therefore I cannot split out the digits/characters and recognize it.

Here is my image:

I have tried to denoise it through using scikit image function.

First, import the libraries:

import cv2
from skimage import restoration
from skimage.filters import threshold_otsu, rank
from skimage.morphology import closing, square, disk

then, I read the image and convert it to gray scale

image = cv2.imread("plate.jpg")
image = cv2.cvtColor(image, cv2.COLOR_BGR2GRAY)

I try to remove the noise:

denoise = restoration.denoise_tv_chambolle(image , weight=0.1)
thresh = threshold_otsu(denoise)
bw = closing(denoise  > thresh, square(2))

What I got is :

As you can see, all the digits are mixed together. Thus, I cannot separate them and recognize the characters one by one.

What I expect is something like this (I draw it):

I am looking for help how can I better filter the image? Thank you.

===================================================================== UPDATE:

After using skimage.morphology.erosion, I got:

Answer 1

First, this image seems to be more defaced by blur, than by noize, so there are no good reasons to denoise it, try debluring instead.

The simplest would be inverse filtering or even Wiener filtering. Then you'll need to separate image's background from letters by the level of luminosity for example with watershed algorithm. Then you'll get separate letters which you need to pass through one of classifiers for example, based on neural networks (even simplistic feed-forward net would be ok).

And then you'll finally get the textual representation. That's how such recognitions are usually made. There's good book by Gonzalez&Woods, try looking for detailed explaination there.

Answer 2

I concur with the opinion that you should probably try to optimize the input image quality.

Number plate blur is a typical example of motion blur. How well you can deblur depends upon how big or small is the blur radius. Generally greater the speed of the vehicle, larger the blur radius and therefore more difficult to restore.

A simple solution that somewhat works is de-interlacing of images.

Note that it is only slightly more readable than your input image. Here I have dropped every alternate line and resized the image to half its size using PIL/Pillow and this is what I get:

from PIL import Image
img=Image.open("license.jpeg")
size=list(img.size)
size[0] /= 2
size[1] /= 2
smaller_image=img.resize(size, Image.NEAREST)
smaller_image.save("smaller_image.png")

The next and more formal approach is deconvolution.

Since blurring is achieved using convolution of images, deblurring requires doing the inverse of convolution or deconvolution of the image. There are various kinds of deconvolution algorithms like the Wiener deconvolution, Richardson-Lucy method, Radon transform and a few types of Bayesian filtering.

You can apply Wiener deconvolution algorithm using this code. Play with the angle, diameter and signal to noise ratio and see if it provides some improvements.

The skimage.restoration module also provides implementation of both unsupervised_wiener and richardson_lucy deconvolution.

In the code below I have shown both the implementations but you will have to modify the psf to see which one suits better.

import numpy as np
import matplotlib.pyplot as plt
import cv2
from skimage import color, data, restoration
from scipy.signal import convolve2d as conv2

img = cv2.imread('license.jpg')
licence_grey_scale = color.rgb2gray(img)

psf = np.ones((5, 5)) / 25

# comment/uncomment next two lines one by one to see unsupervised_wiener and richardson_lucy deconvolution
deconvolved, _ = restoration.unsupervised_wiener(licence_grey_scale, psf)
deconvolved = restoration.richardson_lucy(licence_grey_scale, psf)

fig, ax = plt.subplots()
plt.gray()
ax.imshow(deconvolved)
ax.axis('off')
plt.show()

Unfortunately most of these deconvolution alogirthms require you to know in advance the blur kernel (aka the Point Spread Function aka PSF).

Here since you do not know the PSF, so you will have to use blind deconvolution. Blind deconvolution attempts to estimate the original image without any knowledge of the blur kernel.

I have not tried this with your image but here is a Python implementation of blind deconvolution algorithm: https://github.com/alexis-mignon/pydeconv

Note that an effective general purpose blind deconvolution algorithms has not yet been found and is an active field of research.

Answer 3

ChanVeseBinarize with an image enhanced binarized kernel gave me this result. This is helpful to highlight 4,8,1 and 2. I guess you need to do separate convolution with each character and if the peak of the convolution is higher than a threshold we can assume that letter to be present at the location of the peak. To take care of distortion, you need to do the convolution with few different types of font of a given character.

Another potential improvement using derivative filter and little bit of Gaussian smoothing. The K & X are not as distorted as the previous solution.