2

I'm trying to construct a binary classifier with a neural network on some images using Lasagne. The training and validation loss fluctuate wildly (and do not settle) and the validation accuracy is always at 0%. Furthermore, the network always predicts the target as 1 for the test set.

The network I am using is basically just a copy of Lasagne's example for the mnist dataset found here, but adapted for my images which are quite a bit larger (509 x 115) with around 400 images in the training set. I am wondering if this is a problem, and if the network may need to be deeper / have more neurons?

Do I need a larger training set for this size of image? Or should I be seeing some, albeit inaccurate, set of predictions for my test set?

IVlad
  • 43,099
  • 13
  • 111
  • 179
mjacuse
  • 79
  • 1
  • 5
  • We need a lot more data to be able to answer this: what your task is, what labels you have, how many of each, your complete code. As it stands, this is too broad, unclear and off topic. – IVlad Sep 10 '15 at 17:09
  • If your network always predicts the same class and you get 0% accuracy on the validation set, that sounds like your validation set only contains examples of one class, which would be a bad choice. (Maybe your training set also only contains one class?) For a binary classification task you should get at least 50% accuracy on the validation set. – aleju Sep 11 '15 at 10:41
  • Hi user3760780, thanks for your help. that was indeed the problem, my validation set was all the same class. I have sorted that out now, and the training/val/test sets are all randomised. however, my network still predicts everything as 1 and with a constant 82% accuracy (I assume 82% of the data must be of class '1'). Do you think there needs to be more data of class 0? surely the network must predict at least some class '0' even with this setup? thanks in advance. – mjacuse Sep 11 '15 at 13:34

1 Answers1

4

I'd resize the images into smaller ones. Since your training examples are so limited you probably do not want to train a big model which easily overfits.

The following tricks may also be useful for you:

  • check whether your images are subtracted some mean value. If your input values are raw pixels between [0,255], that will be too big.

  • try different learning rates. If your result fluctuates it is possible your learning rate is too high.

  • use data augmentation. You may flip your images, or move it up/down/left/right some pixels. Then you can get more training examples.

  • look at training set. See where your model makes mistakes. If your training error is bad, the there must be something wrong.

llcao
  • 101
  • 3
  • Hi, thanks for the response. Im going to resize the images in to smaller ones and smaller model. The images are zero meaned and normalised between [0 1]. Im less concerned about the training error, and more concerned that the validation accuracy is always zero and that the classifier predicts the same class for all test inputs. Is this common when training a shallow network on large images? – mjacuse Sep 11 '15 at 09:03
  • @mjacuse I understand training error is not ur concern. But in this case training error can provide useful information for debugging. If all ur training predictions are of the same class, it suggests your problem exists in optimization-- falling into a bad local minimum. If the training error is low but.validation error is high, it suggests your optimization is good but the model suffers from overfitting, or there exists labeling inconsistency. – llcao Sep 11 '15 at 10:20
  • hi llcao, thanks for your help. I have since resized the images to much smaller ones. I have also identified a labelling consistency which I have corrected, however the network still predicts everything as class 1. Could you suggest how I could avoid falling in to a bad local minimum if that is the problem? – mjacuse Sep 11 '15 at 13:36
  • Hmm, interesting. What is your training error? Do you see the training cost went down? How many classes do you have? Does every class contain the same amount of samples? – llcao Sep 11 '15 at 22:40
  • I have two classes with roughly the same amount of data for each class. both the training error and validation error decrease throughout training, however my validation accuracy is always roughly 50% (as is test accuracy) as the net predicts everything as the same class. thanks for your help. – mjacuse Sep 14 '15 at 12:01