1

I'm trying to use Wiener filtering to unblur an out-of-focus image. My application is purely academic, so I don't need a perfect result. However, I'm running into some odd problems and am unsure if I am doing things correctly.

To begin, I set up a camera and took two sets of images. The first set of images were in focus. I began by taking a picture of a very small LED in a completely darkened room. Then I placed a piece of paper directly in front of the LED (which was now off) and took a picture using the flash. Now I defocused the camera and took another set of images in the same manner.

My understanding is that the photograph of the out-of-focus point light source is an experimentally measured point spread function. Thus, I thought it would be easy to de-blur the out-of-focus image using inverse filtering. Well, it turns out it was not so easy after all (see previous post). So now I am trying to implement a Wiener filter and I'm not having a whole lot of luck. Here is my program so far. I have set up two sliders to change the values of kernel_size and restoration_parameter without having to rerun the program.

import numpy as np
import matplotlib.pyplot as plt
import ipywidgets as widgets
import cv2

kernel_size = 5
restoration_parameter = 1

# Read in images: out-of-focus, in-focus, and experimental point spread function
img = cv2.imread('pictures/out_of_focus.jpg')
blur = img[:,:,0]
img2 = cv2.imread('pictures/in_focus.jpg')
clean = img2[:,:,0]
img = cv2.imread('pictures/PSF.jpg')
psf1 = img[:,:,0]

psf2 = np.ones((kernel_size, kernel_size)) / kernel_size**2  # A square kernal
psf = psf2  # set psf to be either psf1 (experimental point spread function) or psf2 (square kernal)
deconvolved_img = restoration.wiener(blur, psf, restoration_parameter, clip=False)

fig = plt.figure()
ax = plt.subplot(111)
new_image = ax.imshow(deconvolved_img)
plt.gray()
plt.show()

def update(kernel_size, restoration_parameter):
    psf2 = np.ones((kernel_size, kernel_size)) / kernel_size**2
    psf = psf2  # set psf to be either psf1
    deconvolved_img = restoration.wiener(blur, psf, restoration_parameter, clip=False)
    new_image.set_data(deconvolved_img)
    ax.set_title(r'kernel size = %2.0f, restoration parameter =%2.5f' % (kernel_size, restoration_parameter))
    return

widgets.interact(update, restoration_parameter=widgets.FloatSlider(min=0,max=100,step=0.1,value=epsilon,description=r'Res. Par.'),
                kernel_size=widgets.IntSlider(min=0,max=40,step=1,value=kernel_size,description=r'kernel size'))

If psf is set to psf1, the program uses the experimentally measured point spread function. In this case, the kernel_size parameter is not used. To my surprise, the un-blurred image just looks like noise, with no hint of the image I was expecting to see. This is very puzzling to me because I thought this was an experimentally obtained point spread function for the system I am dealing with, and as such should be a good starting point. Apparently, I must be mistaken (I was also mistaken to think that inverse filtering would work). It would be much appreciated if anyone could explain to me why this doesn't work at all.

If psf is set to psf2, the program uses a square kernel of size kernel_size. In this case, the deblurred image does resemble what I would expect, but the final image is still completely blurred and no set of parameters seems to help. I guess I'm not surprised this doesn't work all that well because the point spread function here does not seem to be a very good approximation to the real point spread function. But I am surprised that it works better than when using the experimentally measured point spread function. This makes me feel that I may be doing something wrong.

In any event, I am hoping someone can educate me on what I am doing wrong (if anything) and point me in a direction that might lead to some success. By the way, although I do have the in-focus image for reference, I do not want to use it in the calculation. I am trying to demonstrate a real-world scenario where you might have taken an out-of-focus picture and you want to fix it without having access to an in-focus picture.

aTdHvAaNnKcSe

In case it is helpful, here are the out-of-focus image (text) and the out-of-focus LED (what I believe to be an experimental point spread function).

Out-of-focus image

Point spread function

DJElectric
  • 349
  • 1
  • 4
  • 18
  • I had a quick look into this as it seems quite interesting. I was curious if it was even possible (if the information was never there then how can it be recovered), but it somehow seems so. I found a similar question that contains python code, could potentially be helpful - https://stackoverflow.com/questions/35192550/wiener-filter-for-image-deblur – Peter Dec 14 '18 at 15:16
  • The idea is that the information _is_ in the picture, but it has been mixed up with nearby pixels, which is what creates the blur. I have seen a number of examples, but they always deal with images that have been artificially blurred with a known PSF, and I am interested in the more realistic situation where you don't know the PSF in advance. – DJElectric Dec 14 '18 at 15:21
  • May be a dumb suggestion (I'm assuming a different PSF is meant for a different focus), but could you generate multiple images and calculate if it's more or less in focus than the previous one, a bit like how autofocus works? – Peter Dec 14 '18 at 15:38
  • I don't quite understand your question. I actually did take multiple images at different focus settings and one thing I noticed was that the point spread function appeared to be fairly similar for the different settings (larger for the more out-of-focus images). This made me think that I would be able to model the PSF, perhaps as a simple circle of varying radius, and then write a program to vary the size and potentially un-blur any image taken with this camera. But since I can't even get a clear image with an experimental PSF, the odds of being able to model it are extremely small. – DJElectric Dec 14 '18 at 16:00
  • Does `restoration.wiener` come from skimage? (there's no import for it!) – Cris Luengo Dec 14 '18 at 16:54
  • I think you make a mistake in thinking that you experimentally determined the point spread function. You projected the LED onto your camera using a pinhole. You are measuring the size of the LED + the camera's defocus PSF. – Cris Luengo Dec 14 '18 at 16:57
  • You would be better off estimating the PSF using an appropriate target. An edge will do: print out a big black box and photograph that. Then measure the transition between black and white in the image, perpendicular to the edge. This should be the integral of the PSF. You can assume your PSF is a disk, then estimate the diameter from the transition. – Cris Luengo Dec 14 '18 at 16:59
  • Yes, `restoration.wiener` comes from skimage. I am working in a Jupyter notebook and had imported that earlier and forgot to include it in the program above. In regards to the PSF, isn't the camera's defocus PSF what I _want_ to measure? Can you explain more about measuring the PSF using a black square? I don't quite follow your comment that one can assume the PSF is a disk (if I'm assuming it's a disk, why bother measuring it at all?). By the way, I already tried using a simple disk for the PSF and just adjusting its size, but that didn't work either. – DJElectric Dec 15 '18 at 17:14

0 Answers0