2

I have some points that I need to classify. Given the collection of these points, I need to say which other (known) distribution they match best. For example, given the points in the top left distribution, my algorithm would have to say whether they are a better match to the 2nd, 3rd, or 4th distribution. (Here the bottom-left would be correct due to the similar orientations)

Points Given Sample distribution 1 Sample distribution 2 Sample distribution 3

I have some background in Machine Learning, but I am no expert. I was thinking of using Gaussian Mixture Models, or perhaps Hidden Markov Models (as I have previously classified signatures with these- similar problem).

I would appreciate any help as to which approach to use for this problem. As background information, I am working with OpenCV and Python, so I would most likely not have to implement the chosen algorithm from scratch, I just want a pointer to know which algorithms would be applicable to this problem.

Disclaimer: I originally wanted to post this on the Mathematics section of StackExchange, but I lacked the necessary reputation to post images. I felt that my point could not be made clear without showing some images, so I posted it here instead. I believe that it is still relevant to Computer Vision and Machine Learning, as it will eventually be used for object identification.

EDIT:

I read and considered some of the answers given below, and would now like to add some new information. My main reason for not wanting to model these distributions as a single Gaussian is that eventually I will also have to be able to discriminate between distributions. That is, there might be two different and separate distributions representing two different objects, and then my algorithm should be aware that only one of the two distributions represents the object that we are interested in.

hjweide
  • 11,893
  • 9
  • 45
  • 49
  • why not ask that on http://stats.stackexchange.com? – Qnan Aug 20 '12 at 14:46
  • 1
    Surely I would not have enough reputation there to post images? (I do not have enough on Mathematics- where I do have an account) – hjweide Aug 20 '12 at 14:47
  • But where do I host the images then? I suppose I could host it on my university's webspace and post the links. I will do that if I do not find answers here, thanks for the advice! – hjweide Aug 20 '12 at 14:51
  • There's lots of services for that, I believe, e.g. photobucket, tinypic, etc. – Qnan Aug 20 '12 at 14:52

3 Answers3

1

I think this depends on where exactly the data comes from and what sort of assumptions you would like to make as to its distribution. The points above can easily be drawn even from a single Gaussian distribution, in which case the estimation of parameters for each one and then the selection of the closest match are pretty simple.

Alternatively you could go for the discriminative option, i.e. calculate whatever statistics you think may be helpful in determining the class a set of points belongs to and perform classification using SVM or something similar. This can be viewed as embedding these samples (sets of 2d points) in a higher-dimensional space to get a single vector.

Also, if the data is actually as simple as in this example, you could just do the principle component analysis and match by the first eigenvector.

Qnan
  • 3,714
  • 18
  • 15
  • Well, ideally I would need more than one distribution, as the shape of the points are free to vary to look nothing like a single Gaussian. The points come from features extracted from a 2D-view from 3D-models during an object-registration process. Could you please elaborate on "selection of the closest match"? How do I compare Gaussian Distributions for similarity? Mean and Covariance? – hjweide Aug 20 '12 at 14:59
  • For a single Gaussian I think it would be easiest to estimate the likelihood of the points in a new sample with respect to each of the distributions. – Qnan Aug 20 '12 at 15:03
1

You should just fit the distributions to the data, determine the chi^2 deviation for each one, look at F-Test. See for instance these notes on model fitting etc

reptilicus
  • 10,290
  • 6
  • 55
  • 79
1

You might want to consider also non-parametric techniques (e.g. multivariate kernel density estimation on each of your new data set) in order to compare the statistics or distances of the estimated distributions. In Python stats.kde is an implementation in SciPy.Stats.

gevang
  • 4,994
  • 25
  • 33