0

Since the implementation of the algorithm is correct(i checked it hundreds of times), I think I have misunderstood some theorical facts.
I suppose that:
given that j refers to the hiddenlayer side and k to the output layer
∂E/∂wjk is calculated by doing: 

outputNeuron[k].errInfo=(target[k]-outputNeuron[k].out)*derivate_of_sigmoid(outputNeuron[q].in);
∂E/∂wjk=outputNeuron[k].errInfo*hiddenNeuron[j].out;

For the ∂E/∂wij, where 'i' refers to the inputlayer and 'j' to the hiddenlayer, it's a bit longer.
Each hidden unit (Zj, j = 1, ... ,p) sums its delta inputs (from units in the output layer),

errorInfo_in[j]=summation from k=1 to m(number of output units) of: outputNeuron[k].errInfo*w[j][k]

Then i calculate the error info of the hidden unit:

hiddenNeuron[j].errInfo=errorInfo_in[j]*derivated_sigmoid(hiddenNeuron[j].in);

Finally the ∂E/∂wij is:

hiddenNeuron[j].errInfo*x[i]   (where x[i] is the output of an input unit)

I apply the RPROP as described here http://www.inf.fu-berlin.de/lehre/WS06/Musterererkennung/Paper/rprop.pdf
For all the weight between the input and hidden, and hidden output.
I'm trying to recognize letters made of '#' and '-' , 9(rows)x7(columns).
The MSE just get stuck at 172 after a few epoch.
I know that RPROP is a batch learning, but i'm using online learning because i read that it works anyway.

Lopo Kima
  • 1
  • 1

1 Answers1

1

RPROP does not work well with pure online learning, it might work with mini-batch learning, provided mini-batch is large enough. Absolute value of MSE is poor indicator of anything, especially on custom datasets where gold standard values are unknown.

It is best to test newly implemented NN algorithms on simple things like logic gates (AND, OR, XOR), before moving to something more complex. This way, you will always be confident in your code and methodologies. For character recognition tasks, you may also want to test on well known datasets such as MNIST, where expected results are known and you can compare your results to previous work.

For classification tasks one usually want to measure accuracy of classification as it is much better perfomance indicator than MSE.

Denis Tarasov
  • 1,051
  • 6
  • 8