3

I am working on an application that should determine if input image contain a stamp imprint and return its location. For RGB images I am using color segmentation and doing verification (with various shape factors), for grayscale image I thought that SIFT + verification would do the job, but using SIFT would only find those stamps(on input image) that I got in my database.

In ideal case it works really well, as shown on image bellow.

Fig. 1. https://i.stack.imgur.com/JHkUl.png

The problem occurs when input image contains a stamp that does not exist in database. First thing I did was checking if there would be any matching key points if I compare a similar stamp to the one on input image. In most cases there is no single matching key point and if there is some they rather refer to other parts of input image than a stamp, as shown in Fig. 2.:

Fig. 2. https://i.stack.imgur.com/coA4l.png

I also tried to find a match between input and circle images as the stamps are circular, but circle image has very few key points, if any.

So I wonder if there is any different approach that will make SIFT a bit more useful in this exact case? I though about creating a matrix with all descriptors and key-points from my database and then looking for nearest euclidean distance between input image and matrix, but it probably wont work as there is a lot of matching key-points(unwanted) across the database (see Fig. 2.).

I'm working with Matlab and tried both VLFeat and D. Lowe SIFT implementations.

Edit:

So I found a way to force SIFT to compute descriptors for user defined points on an image. My test image contained a circle, then the descriptors were computed and matched against input images, including the one under Fig 1 and 2. This process was repeated for scales from 0 to 10. Unfortunately it didn't help too.

sowizz
  • 175
  • 3
  • 11
  • If all your stamps are round, I would actually start with a Hough transform to find circles. Then you can take just the candidate areas and compare them to your actual stamp images. I imagine you might be able to train an SVM or something to deal with it. – Nicu Stiurca Aug 29 '13 at 19:27

2 Answers2

1

This is only a first hint and not a full answer to the SIFT questions.

My impression is that detecting a circle by matching it against an image of a circle via SIFT is not the best approach, especially if the circle you want to detect has some unknown texture inside.

The textbook algorithm for circle detection would be Hough transform, which is mostly used for line detection but does work for any kind of shape which can be described by a low number of parameters (colleagues tell me things get nasty above 3, but a circle just has X,Y and r). There are several implementations in file exchange, the link is just to one example. Hough circle detection requires you to put an upper bound on the radii you want to detect, but this seems ok for your application.

From the examples you provided it looks like you should get quite far if you can detect circles reliably.

DCS
  • 3,354
  • 1
  • 24
  • 40
  • Thanks for your answer. I know about Hough transform, but the original idea was to use SIFT as it is scale-invariant in contrast to Hough transform, which make it resource hungry. However if I fail to come up with any idea I am going to use Hough transform, I would also give Harris corner detector a go. Another disadvantage of Hough transform is that I am detecting also square, rectangle, triangle, oval stamps. There is also a group of irregularly shaped stamps so there is no way they could be described. – sowizz Aug 02 '12 at 13:00
  • I get your point - you want to know whether SIFT can do the job. From my experience with SIFT I think it probably cannot, at least not the circle or shape detection part. This is simply not what SIFT is for: It aggregates local properties of texture (in a way which is invariant under certain transformations), but it is unaware of geometric entities like circles etc. If you want to detect the unknown patterns with SIFT I thinks you have to use a full machine learning approach, feeding the SIFT descriptors, for example, into an SVM which you train on your known patterns. – DCS Aug 07 '12 at 14:41
0

Actually I do not think SIFT will be solving this problem. I've been playing around with SIFT for quite some time and my conclusion is that it's really great for identifying identical patterns but not for similar patterns. Just have a look at the construction of the SIFT feature vector: The descriptor is composed of several histograms of gradients(!). If you have patterns in the database that have very similar blob like structures in the stamps, then you might have a chance. But if this does not hold, then I guess you will not be very lucky.

From my point of view you have kind of solved the problem of finding indentical objects (stamps) and now extend to finding similar objects. This sounds like the same but in my past research I found these problems just related but not too identical.

Do you have any runtime constraints in your application? There might be other approaches but in this case, more input about possible constraints might be useful.


Update regarding constraints:
So your next task might be to detect the unknown stamps, right? This sounds like a classification task.

In your case I would first try to find a descriptor/representation (or SVM) that classifies images into stamp/no-stamp. In order to evaluate this, set up a data base with ground truth and a reasonable amount of "unknown" stamps and other images like random snapshots from the letters, NOT containing stamps. This will be your test set.

Then try some descriptors/representations to caluclate the distance/similarity between your images to classify your test set into the classes STAMP / NO-STAMP. When you have found a descriptor/distance measure (or SVM) that performs well in classifying, then you could perform a sliding window approach on a letter to find a stamp. The sliding window approach is certainly not a very fast method, but a very easy one. At least when you have reached this point, you can tune the detection - for example based on interesting point detectors.. but one step after the other...

Locked
  • 186
  • 13
  • You're right about SIFT, it is not applicable in my work. I tried to extort key-points for which SIFT calculates features vector, and put them on edges of stamps, but SITF is texture-only sensitive and it won't work like this. That's why I gave up on it and used circular Hough, Hough for lines and other approach, suggested [here](http://stackoverflow.com/questions/12158396/how-to-check-if-four-points-form-a-rectangle). – sowizz Oct 18 '12 at 21:54
  • To answer your question, I do not have any runtime constranst, as it is part of my Bsc thesis, but on other hand I can't go extreme. If you have any suggestion please share, it would be much appreciated. – sowizz Oct 18 '12 at 21:55