1

I'm trying to implement the Bag of Features model.

Given a descriptors matrix object (representing an image) belonging to the initial dataset, compute its histogram is easy, since we already know to which cluster each descriptor vector belongs to from k-means.

But what about if we want to compute the histogram of a query matrix? The only solution that crosses my mind is to compute the distance between each vector descriptor to each of the k cluster centroids.

This can be inefficient: supposing that k=100 (so 100 centroids), then we have an query image represented through 1000 SIFT descriptors, so a matrix 1000x100.

What we have to do now is computing 1000 * 100 eucledian distances in 128 dimensions. This seems really inefficient.

How to solve this problem?

NOTE: can you suggest me some implementations where this point is explained?

NOTE: I know LSH is a solution (since we are using high-dim vectors), but I don't think that actual implementations use it.

UPDATE:

I was talking with a collegue of mine: using a hierarchical cluster approach instead of classic k-means, should speed up the process so much! Is it correct to say that if we have k centroids, with an hierarchical cluster we have to do only log(k) comparisons in order to find the closest centroid instead of k comparisons?

justHelloWorld
  • 6,478
  • 8
  • 58
  • 138
  • You're right, for each of the `1000` features you need to find their corresponding centres to build the histogram. That's 1000 nearest-neighbour queries per query matrix. Dealing with NN are kd-trees, random forests, product quantization and LSH to name a few. Each have their strenghts and weaknesses. – BeyelerStudios Jun 22 '16 at 11:44
  • Thanks for your comments. I think that you're right and I think that the main point about the solutions that you proposed is about vector dimensionality and speed. For example LSH is for high dimensionality in a fast (and approximate) way. Do you agree? In addition, do you know some BoF implementations in high dimensions space? – justHelloWorld Jun 23 '16 at 06:18
  • 1
    Kd-trees and random forests are both exact solutions while product quantization can be and LSH is an approximation. These methods are usually combined (e.g. product quantization and kd-tree) to solve problems like yours. For implementations follow the research, I'd need to google, just like you. – BeyelerStudios Jun 23 '16 at 07:24
  • Could you please give a look a [this](http://stackoverflow.com/questions/37987863/similar-images-bag-of-features-visual-word-or-matching-descriptors) question? – justHelloWorld Jun 23 '16 at 09:46
  • Please, look at the **UPDATE** section :) – justHelloWorld Jun 23 '16 at 11:33

1 Answers1

1

For a bag of features approach, you indeed need to quantize the descriptors. Yes, if you have 10000 features and 100 features that 10000*100 distances (unless you use an index here). Compare this to comparing each of the 10000 features to each of the 10000 features of each image in your database. Does it still sound that bad?

Has QUIT--Anony-Mousse
  • 76,138
  • 12
  • 138
  • 194
  • Thanks for your answer. By "quantize the descriptors" I think you mean reducing its dimension, speeding up matching operation. From what I know, LSH is the most (only?) solution for this approach. Anyway, what do you mean by "unless you use an index here"? In addition, do you know some BoF implementation for high dimension vectors (like SIFT) which exploit quantization? – justHelloWorld Jun 23 '16 at 06:24
  • Could you please give a look a [this](http://stackoverflow.com/questions/37987863/similar-images-bag-of-features-visual-word-or-matching-descriptors) question? – justHelloWorld Jun 23 '16 at 09:46
  • Please, look at the **UPDATE** section :) – justHelloWorld Jun 23 '16 at 11:33
  • Quantization is more appropriate than "clustering" for this use case of k-means (quality is not too important). Hierarchical clustering is *much* more expensive than k-means. – Has QUIT--Anony-Mousse Jun 23 '16 at 14:09
  • In term of time to obtain the cluster yes, but what about for creating the histogram? – justHelloWorld Jun 23 '16 at 14:36