2

I implemented a neural network for AND gate with 2 input units, 2 hidden units and 1 output unit. I trained the neural network using 40 inputs for 200 epochs with a learning rate of 0.03. When I try to test the trained neural network for AND inputs, it gives me output as :

  1. 0,0 = 0.295 (0 expected)
  2. 0,1 = 0.355 (0 expected)
  3. 1,0 = 0.329 (0 expected)
  4. 1,1 = 0.379 (1 expected)

This is not the output which is expected from the network. But if I set the threshold as 0.36 and set all values above 0.36 as 1 and rest as 0, the neural network output is just as expected every time. My question is that : Is applying a threshold to the output of the network necessary in order to generate expected outputs like in my case ?

pr22
  • 179
  • 1
  • 2
  • 9
  • Hi, you are not supposed to apply a threshold to the output of the network. You have to set all weights random (including threshold of each neuron), then give it epochs / supposed result and compute error (difference between supposed result and network output) then from this error you have to use *backpropagation* (https://en.wikipedia.org/wiki/Backpropagation) to adjust each weights of the network. After some epoch, every neuron will adjust its weights (including threshold) to give the right output from the given inputs. – Sebastien Servouze May 17 '18 at 07:43
  • Sorry for double comment, had not enough space, but here is a **full mathematical backpropagation demonstration** you should be able to reproduce by code: https://mattmazur.com/2015/03/17/a-step-by-step-backpropagation-example/ – Sebastien Servouze May 17 '18 at 07:45

1 Answers1

2

A threshold is not necessary, but can help better classification, for example.

In your case, maybe you can set a threshold of 0.1 for 0, and 0.9 for 1. When your output is below 0.1, we can consider that it's a 0, and if output is higher than 0.9, then it's a 1.

Therefore, set a threshold of 0.36 just because with that test example it works, is a really bad idea. Because 0.36 is far from the output 1 that we want. And because it may (will) not work with all your test datas.

You should consider problem with your code.

This is not the initial question, but here is some ideas :
1. Look at you training accuracy at each eopch. If it's learn slowly, increase your learning rate, and maybe descrease it after few epoch.
2. If the accuracy don't change, look at your Back probagation algorithm
3. Look at your dataset and be sure inputs and outputs are correct
4. Be sure that your weights are initialized randomly
5. AND gate can be solve with a linear NN, without hidden layer. Myabe try removing your hidden layer ?

Jérémy Blain
  • 249
  • 2
  • 14