Dust and scratch removal with open source graphic libraries

Question

I am trying to automate the cleanup process of a large amount of scanned films. I have all the images in 48-bit RGBI TIFF files (RGB + Infrared), and I can use the infrared channel to create masks for dust removal. I wonder if there is any decent open source implementation of in-painting that I can use to achieve this (all the other software I use for batch processing are open source libraries I access through Ruby interfaces).

My first choice was ImageMagick, but I couldn't find any advanced in-painting option in it (maybe I am wrong, though). I have heard this can be done with MagickWand libraries, but I haven't been able to find a concrete example yet.

I have also had a look at OpenCV, but it seems that OpenCV's in-paint method accept only 8-bit-per-channel images, while I must preserve the 16. Is there any other library, or even an interesting code snippet I am not aware of? Any help is appreciated.

Samples:

Full Picture

IR Channel

Dust and scratch mask

What I want to remove automatically

What I consider too large to remove with no user intervention

You can also download the original TIFF file here. It contains two alpha channels. One is the original IR channel, and the other one is the IR channel already prepared for dust removal.

Answer 1

I have had an attempt at this, and can go some way to achieving some of your objectives... I can read in your 16-bit image, detect the dust pixels using the IR channel data, and replace them, and write out the result without any alpha channel and all the while preserving your 16-bit data.

The part that is lacking is the replacement algorithm - I have just propagated the next pixel from above. You, or someone cleverer than me on Stack Overflow, may be able to implement a better algorithm but this may be a start.

It is in Perl, but I guess it could be readily converted to another language. Here is the code:

#!/usr/bin/perl
use strict;
use warnings;
use Image::Magick;

# Open the input image
my $image = Image::Magick->new;
$image->ReadImage("pa.tiff");
my $v=0;
# Get its width and height
my ($width,$height)=$image->Get('width','height');

# Create output image of matching size
my $out= $image->Clone();

# Remove alpha channel from output image
$out->Set(alpha=>'off');

# Load Red, Green, Blue and Alpha channels of input image into arrays, values normalised to 1.0
my (@R,@G,@B,@A);
for my $y (0..($height-1)){
   my $j=0;
   my @RGBA=$image->GetPixels(map=>'RGBA',height=>1,width=>$width,x=>0,y=>$y,normalize=>1);
   for my $x (0..($width-1)){
      $R[$x][$y]=$RGBA[$j++];
      $G[$x][$y]=$RGBA[$j++];
      $B[$x][$y]=$RGBA[$j++];
      $A[$x][$y]=$RGBA[$j++];
   }
}

# Now process image
my ($d,$r,$s,@colours);
for my $y (0..($height-1)){
   for my $x (0..($width-1)){
      # See if IR channel says this is dust, and if so, replace with pixel above
      if($A[$x][$y]<0.01){
         $colours[0]=$R[$x][$y-1];
         $colours[1]=$G[$x][$y-1];
         $colours[2]=$B[$x][$y-1];
         $R[$x][$y]=$R[$x][$y-1];
         $G[$x][$y]=$G[$x][$y-1];
         $B[$x][$y]=$B[$x][$y-1];
         $out->SetPixel(x=>$x,y=>$y,color=>\@colours);
      }
   }
}

$out->write(filename=>'out.tif',compression=>'lzw');

The result looks like this, but I had to make it a JPEG just to fit on SO:

Answer 2

I cannot comment, so I write an answer.

I suggest using G'Mic with the filter "inpaint".

You should load the image, take the IR image and convert it to b/w, then tell the filter inpaint to fill the areas marked in the IR image.

Answer 3

OpenCV has a good algorithm for image inpaiting, which is basically what you were searching for. https://docs.opencv.org/3.3.1/df/d3d/tutorial_py_inpainting.html

If that will not help then only Neural Networks algorithms