Combining multiple features vector, HOG and LBP

Question

I am working on people detection using two feature descriptor, HOG and LBP. So far, I combine both of the features using a simple concatenation. But it shows me sometimes problem due to big vectors. Here is my code.

%extract features from negative and positive images

[HOGpos,HOGneg] = features(pathPos, pathNeg);  


% loading and labeling each training example
HOG_featV = HOGfeature(fpos,fneg);   


% get label of training data from HOG
HOGlabel  = cell2mat(HOG_featV(2,:));

% get the feature vector value from HOG
HOGfeatureVector = HOG_featV(3,:)';

C = cell2mat(HOGfeatureVector); % each row of P correspond to a training example 




%extract features from LBP
[LBPpos,LBPneg] = LBPfeatures(pathPos, pathNeg);


% loading and labeling each training example
LBP_featV = loadingV(LBPpos, LBPneg); 


% get label of training data from LBP
LBPlabel = cell2mat(LBP_featV(2,:));

% get feature vector value from LBP
LBPfeatureVector = LBP_featV(3,:);
M = cell2mat(LBPtP)'; % each row of P correspond to a training example


%concatenate HOG and LBP feature
featureVector = [C M];

I want to know, is there any method to combine two feature vector which is more reliable and faster? If yes, please give some suggestion or link that I can refer. Thank you.

Answer 1

I understood from you comments the following: You are selecting 1845 key points in the image. For each point you calculate a feature vector of length 383 (LBP+HOG combined). The total vector that represents the image is of length ~100,000.

If in fact you have only 1845 images and each image is represented by only 383 features than you are doomed to fail and your SVM will have a very high error rate. Mainly because (the features vector is too short and the amount of training images is too small). So I assume that is not the case

You have a few problems in your approach.

1. It seems you don't understand the difference between detector and classifier, and you are trying to apply classifier to solve detection problem. There is a core difference. Classifier is designed to distinguish between 2 (or more) classes. For example is this fruit and 'apple' or an 'orange'. You train it by giving a lot of examples of both classes. But if you give a Banana to the the classifier it will will return a random! result. It might say 'apple' with 100% confidence or an orange with 100% confidence. Classifier cannot deal with examples (like banana) that are not within the training classes (apple, orange). On the other hand detector doesn't have 2 classes. It has an 'apple' and 'everything else'. There are no examples outside of the training classes because 'apples' and 'everything else' cover the entire world. You can not train a classifier to recognize 'everything else' because there is an infinite variety of objects that are not 'apple'. 'Everything else' is not a class that can be trained to recognize by classifier. For that you need a detector. The training process of the detector is a bit different and the types of examples you supply are different. To train a detector you have to supply a huge amount of 'non apples. Images of cars, human beings, chairs, airplains. Typically you need many millions of different examples of 'non apples' and only a few examples of apples (say few thousands).
2. SVM is not a good solution for detection problem. the core idea of the SVM is that you can select few representative examples (support vector) and describe the separation between 2 classes using those representatives. But let me ask you - what are the representative images of everything that is 'not an apple'? It is impossible to answer that question. The amount of objects that are not 'apples' is infinite so you cannot select good support vectors for them. It is easy to select representative to 'apple' (image of red apple, green apple, rotten apple, etc) but not to the 'everything else' class. Don't get me wrong - People use SVM for detection of easy objects (like corporate logos) but when your task is difficult like people detection (you have a huge intra-class variation) - SVM is inferior. Even if you manage to train an SVM for detection you will have 2 problems: Slow run time, high error rate on new examples (due to intra-class variation). I would suggest to use boosting methods to train a detector.
3. I assume you get a high error rate because the amount of images you use for training is very low. Your feature vector is of length ~100,000 so if you use few millions of images for training - it will take few days for the SVM to get trained. Remember: for 2 class - classification problems few thousands of images might be enough. But, for detection problems you need millions of images.
4. Your amount of features is too high (assuming it is 100,000). I am betting that you have less image samples than ~100,000 length of your features vector. For detection problems the amount of features should be relatively low (300-3000) but each feature must be good! For example you start with a list of 60,000 different features and during training process reduce the number to say 1,500 good feature and use only them for detection. If you start with lower amount of features (like 383 = LBP histogram + HOG histogram) than most of the features are useless and if you do PCA you might be surprised that effectively you have only ~40 features that help the detection and the rest are useless. On the contrary if you finish you training still requiring 100,000 features you will get crazy over-fitting wich will cause a high error rate of the SVM.

Practical advise:

1. You can keep working with LBP/HOG features but first make sure you have a huge amount of 'non people' examples (1-10 millions) Otherwise you will have a high error rate. Try to get also few thousands images of people

2. Train a single cascade detector. It is easier to understand, much faster and is more accurate than SVM. Give the training process all possible HOG, LBP features (without histograms) and let it select the good features. There are free implementations of cascade detector training inside openCV which you can use. My personal advise: do not use SVM for detection. It is a wrong tool. Once you detected a person in the image you can use SVM to classify if it is an adult or a kid but do not use SVM for detection.