4

In brief, what are the available options for implementing the Tracking of a particular Image(A photo/graphic/logo) in webcam feed using OpenCv?In particular i am trying to collate opinion about the following:

  1. Would HaarTraining be overkill(considering that it is not 3d objects but simply Images to be tracked) or is it the only way out?

  2. Have tried Template Matching, Color-based detection but these don't offer reliable tracking under varying illumination/Scale/Orientation at all.

  3. Would SIFT,SURF feature matching work as reliably in video as with static image comparison?

Am a relative beginner to OpenCV , as is evident by my previous queries on SO (very helpful replies). Any cues or links to what could be good resources for beginning NFT implementation with OpenCV?

imeht
  • 67
  • 2
  • 7

2 Answers2

8

Can you talk a bit more about your requirements? Namely, what type of appearance variations do you expect/how much control you have over the environment. What type of constraints do you have in terms of speed/power/resource footprint?

Without those, I can only give some general assessment to the 3 paths you are talking about.

1. Haar would work well and fast, particularly for instance recognition.

Note that Haar doesn't work all that well for 3D unless you train with a full spectrum of templates to cover various perspectives. The poster child application of Haar cascades is Viola Jones' face detection system which is largely geared towards frontal faces (can certainly be trained for many other things)

For a tutorial on doing Haar training using OpenCV, see here.

2. Try NCC or better yet, Lucas Kanade tracking (cvCalcOpticalFlowPyrLK which is a pyramidal as in coarse-to-fine LK - a 4 level pyramid usually works well) for a template. Usually good upto 10% scale or 10 degrees rotation without template changes. Beyond that, you can have automatically evolving templates which can drift over time.

For a quick Optical Flow/tracking tutorial, see this.

3. SIFT/SURF would indeed work very well. I'd suggest some additional geometric verification step to remove spurious matches.

I'd be a bit concerned about the amount of computational time involved. If there isn't significant illumination/scale/in-plane rotation, then SIFT is probably overkill. If you truly need it, check out Changchang Wu's excellent SIFTGPU implmentation. Note: 3rd party, not OpenCV.

peakxu
  • 6,667
  • 1
  • 28
  • 27
  • The Appearance would typically vary in terms of rotation, illumination. Reg HAAR- went through the Viola Jones detector method, tried out the OpenCV bundled sample. What deters me is the time required for training- I work on a 2GB Desktop/Notebook & conflicting views of people regarding it being less than robust even after the days and days of training. I'l go through the Optical Flow tutorial for sure.Thanks for the useful links throughout the reply. – imeht May 11 '11 at 04:14
  • You're quite welcome. Agreed about the Haar training time which is substantial. The upside is that runtime performance can be good. I'm partial to method 2 personally. Method 3 is more for unconstrained scenarios like PhotoTourism. – peakxu May 12 '11 at 12:23
1

It seems that none of the methods when applied alone could bring reliable results unless it is a hobby project. Probably some adaptive algorithm would be more or less acceptable. For example see a famous opensource project where they use machine learning.

Andrey Sboev
  • 7,454
  • 1
  • 20
  • 37
  • Yes i've gathered this from a lot of my trials and other sources- no one method takes care of everything that can vary from scene to scene. Could you point me towards some resource of integrating machine learning with openCV- quite a relative beginner am I. – imeht May 11 '11 at 04:27
  • I tried opencv tracking methods too and I'm going to dive into more complicated stuff and start with learning mentioned technology. Besides that I have nothing to suggest yet. As of integrating machine learning with OpenCV a good news that OpenCV already contains ml module – Andrey Sboev May 11 '11 at 06:07