4

My dataset has 400 observations of 100 features. Each of the 400 observations belongs to 1 out of 2 classes.

I am training a neural network (patternet(15)) in MATLAB for classifying this dataset.
I don't use all the features at once, but first I use one feature (400x1), then I add a second one (400x2) and so on. At every step I calculate the AUC using perfcurve.

Here is my problem:

The AUC is changing but it is not always getting larger with each additional input.
Shouldn't the AUC be increasing as I use more features to train the network? (I always use the same divisions through divideind).

All comments and help are appreciated! Thanks!

glin yon
  • 99
  • 1
  • 1
  • 8
  • It would be helpful to see the ROC curve you produced, as well as the structure of your network. Better yet, you could post the code you used. – bogatron Nov 06 '13 at 14:22
  • 1
    Maybe the first features are more discriminatives, and the last ones only add noise ! You could try to run a random forest and see which features are the most discriminatives between the categories – bendaizer Nov 08 '13 at 16:51
  • @bendaizer you were right, the first ones were the most discriminative! the features I had, had been pre-ordered based on the difference of their distribution between the categories. – glin yon Jan 13 '14 at 16:57
  • @bogatron sorry for taking so much time to answer, I had changed projects for a few months. still it will be difficult to post all the ROC curves, because MATLAB gives me four ROC plots for every additional feature, so they would be about 400 plots – glin yon Jan 13 '14 at 17:02
  • also, I don't know why, but I cannot upvote your comments... – glin yon Jan 13 '14 at 17:02
  • If you want, I can post my comment as an answer and you can vote for it :) – bendaizer Jan 14 '14 at 09:43
  • @bendaizer. Please do. It was the right answer after all :) – glin yon Jan 19 '14 at 17:57

1 Answers1

2

Maybe the first features are more discriminatives, and the last ones only add noise ! You could try to run a random forest and see which features are the most discriminatives between the categories

bendaizer
  • 1,235
  • 9
  • 18