2

I'm trying to train ANN to predict probabilities of an image belonging to a several number of classes, and my target values are sets of such probabilities.

Input is simple reshaped 28x28 grayscale pictures with pixel values from 0-255.

One 'target' looks like this: 0.738832,0.238159,0.023009,0,0.238159,0,0.238159,0,0.238159,0,0,0.238159,0,0.19793,0.80207,0.066806667,0.663691308,0.008334764,0,0,0.0494825,0.098965,0.0494825,0,0,0,0,0,0,0,0,0,0,0,0,0,0

However the results that I get are terrible (far worse than simple linear regression) and looks like this: 0.011947,0.448668,0,0,0.095688,0,0.038233,0,0,0,0,0,0,0,0.405464,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0

It doesn't really matter if I use 300 or 30000 pictures. I'm obviously doing something wrong and I really appreciate some advice.

Code:

# create dataset
DS = SupervisedDataSet(784, 37)
assert(ia.shape[0] == ta.shape[0])
DS.setField('input', ia)
DS.setField('target', ta)

fnn = buildNetwork( DS.indim, 200, 37, outclass=SoftmaxLayer )

trainer = BackpropTrainer( fnn, dataset=DS, momentum=0.1, verbose=True, weightdecay=0.01)
trainer.trainUntilConvergence(maxEpochs=10,verbose=True,validationProportion=0.20)
user2207055
  • 91
  • 1
  • 4

1 Answers1

1

Your problem is the values you are using for training. A softmax layer means that all values for that layer will sum to one. So when you setup 37 output dimensions that means all 37 dimensions will sum to 1.0. Your sample target does not appear to follow that distribution.

Firestrand
  • 1,305
  • 10
  • 19
  • Thank you. As 3 first values are summing up to 1, I've checked it only against them and results are much better. However, I'm wondering if there is a way to predict all the values using single network. – user2207055 Mar 04 '14 at 09:07
  • With pybrain I am not aware of a way to setup and train multiple softmax output layers. You could use a different type of output layer that isn't constrained in the same way. – Firestrand Mar 04 '14 at 20:04