0

I read an idea for a project in the book "Introduction to Machine Learning" by Tom Mitchell. The project is about determining the direction of the face (left, right, down, straight). I use my own developed neural network, which works (tested with XOR, parabola function...), but can't train it enough well to determine them correctly. The best case I got is 43% correct, which is pretty low.

Here is a description of the project:

Images 32 x 30, grey-scale (I use 13 people x 32 images for training examples and 4 poeple x 32 images for tests).

Neural Network: 3 layers - input, hidden, output

32 x 30 input units

3 hidden units, using Sigmoid as a transfer function

1 output unit, using linear as a transfer function.

OUT: 0.2 = left ; 0.4 = down; 0.6 = right; 0.8 straight

Learning rate = Momentum = 0.3

Weights and biases are set to random small values.

After 25000 iterations, still, I have just ~40% correct. In the book they managed to get 90% accuracy!

Any ideas?

Мартин Радев
  • 134
  • 10
  • Do you use backwards propagation to adjust the weights? Is it post to publish the code you use? – Maarten Kesselaers Sep 05 '12 at 17:35
  • 3
    I guess 3 hidden units is not enough. Hidden layer translates signal from input to output layers, and 3 units mean only 3 variables describing image. Also please clarify this: "I use 13x32 images for training examples and 4x32 for tests". Do you mean that you use images of different size? If not, what these numbers mean? And last question: using float numbers (0.2 = left, 0.4 = down) was suggested in the book or it is your own idea? – ffriend Sep 05 '12 at 17:52
  • 13 people * 32 images per person for training examples 4 people * 32 images per person for tests 32 images = 8 left; 8 right; 8 straight; 8 down some people are with glasses – Мартин Радев Sep 05 '12 at 17:59
  • Yes, I propagate backwards to adjust the weights : ) – Мартин Радев Sep 05 '12 at 18:01
  • Could you clarify which of the aspects of your network design came from the book and which you chose on your own? Particularly, in the book's 90% accurate implementation, did the author specify the same learning rate you are using? Did he specify the same linear output function using 0.2 = left, etc.? – Eric G Sep 05 '12 at 18:08
  • The only difference is the output. He uses for output neurons. For example, a "left" image should output 0.9 0.1 0.1 0.1. That's the only difference. Currently, i'm trying with 5 hidden neurons and it got to ~60% accuracy. I will try now with 20 neurons and then with 40. – Мартин Радев Sep 05 '12 at 18:22
  • 1
    Use 4 neurons and select the one with the largest output - this should increase accuracy a lot. – ffriend Sep 05 '12 at 18:41

1 Answers1

1

After the comment of @ffriend everything worked like a charm. I used 4 output neurons and got more than 90% accuracy. If I use more neurons in the hidden layers, the errors get smaller, but the program needs more time to run through the network and back-propagate.

Мартин Радев
  • 134
  • 10