1

I'm learning ANN, I did two script (in fortran90 and python) for simple binary classification problem.

I first did without bias, and I get a good convergence. But adding a bias for each node it does not converge anymore (or everything is going near to 0 or everything near to 1)

The bias is 1 and has a specific weight for each node. It is randomly initialized and then update adding delta such as others weights. I have tried to change gradient step size but it still doing the same thing.

Someone has any clues ?....

EDIT :

The network :

    IN                HIDDEN             OUTPUT node
(each column is a     LAYERS          (each column is
training data)   (2layers of 3node)   the wanted result)


          W1      .___W2__.    W3
|0|0|1|1|-------->|___|___|______
|0|1|0|1|--\/_-\->|___|___|______\_--> |1|1|0|0|
|1|0|1|1|--/\__/->|___|___|______/

The activation function is a sigmoid (1/(1+exp(-x)))

The weight are initialized with a normal distribution in range of [-1, 1]

Dadep
  • 2,796
  • 5
  • 27
  • 40
  • This might not be answerable without a lot more context: e.g., what activation function you are using, how you are initializing the weights (what distribution?), what optimization procedure you are using to train, what parameters you've chosen for the optimization procedure, what your network architecture is (you included more information on CS.SE which you should add here...). I suggest editing the question to include that information, and also show code of a [MCVE](http://stackoverflow.com/help/mcve). I don't understand how a hidden layer can have 3 nodes but only one output. – D.W. May 19 '17 at 18:00
  • I have edit my post and delete on CS.SE – Dadep May 19 '17 at 18:20

1 Answers1

1

you may have problem :

https://datascience.stackexchange.com/questions/15602/training-my-neural-network-to-overfit-my-training-dataset

you should also be careful of your learning step, if it's too large you can't converge.

duntel12
  • 103
  • 5
  • Link only answers aren't encouraged on this site. Better to summarize he relevant bits of the answer here. – MrFlick Aug 22 '17 at 20:36