Understanding of threshold value in a neural network

Question

Consider the hypothetical neural network here

$o_1$ is the output of neuron 1.  
$o_2$ is the output of neuron 2.  
$w_1$ is the weight of connection between 1 and 3.   
$w_2$ is the weight of connection between 2 and 3.  
So the input to neuron 3 is $i =o_1w_1 +o_2w_2$   
Let the activation function of neuron 3 be sigmoid function.  
$f(x) = \dfrac{1}{1+e^{-x}}$ and the threshold value of neuron 3 be $\theta$.  
Therefore, output of neuron 3 will be $f(i)$ if $i\geq\theta$ and $0$ if $i\lt\theta$.

Am I correct?

Please someone edit the above . It says that I am using a code and I had to indent it because it was not allowing me to post. Also the math is not displaying properly — idpd15, Aug 07 '18 at 15:36

score 1 · Answer 1 · answered Aug 07 '18 at 15:45

Thresholds are used for binary neurons (I forget the technical name), whereas biases are used for sigmoid (and pretty much all modern) neurons. Your understanding of the threshold is correct, but again this is used in neurons where the output is either 1 or 0, which is not very useful for learning (optimization). With a sigmoid neuron, you would simply add the bias (previously the threshold but moved to the other side of the equation), so you're output would be f(weight * input + bias). All the sigmoid function is doing (for the most part) is limiting your output to a value between 0 and 1

After a long search, I finally found the answer to my question. I was wondering why some issues have a threshold, some don't, and some have bias. But I want a source for this information, a book or a magazine, to include it in my master thesis. Thank you — mody, Apr 18 '22 at 19:46

score 0 · Answer 2 · answered Aug 07 '18 at 15:42

0

I do not think it is the place to ask this sort of questions. You will find lot of NN ressources online. For your simple case, each link has a weight, so basicly the input of neuron 3 is :

Neuron3Input = Neuron1Output * WeightOfLinkNeuron1To3 + Neuron2Output * WeightOfLinkNeuron2To3 + bias. Then, to get the output, just use the activation function. Neuron3Output = F_Activation(Neuron3Input)

O3 = F(O1 * W1 + O2 * W2 + Bias)

answered Aug 07 '18 at 15:42

Sami Tahri

1,077
9
19

Please don't mind but I literally searched a lot on internet for examples but I couldn't find anything where explicit calculations were done – idpd15 Aug 07 '18 at 15:44
@AdamJohnston Thanks! – idpd15 Aug 07 '18 at 15:50
You generaly use matrices to represent a layer, then multiply matrices and add the bias matrices too. – Sami Tahri Aug 07 '18 at 15:50

Understanding of threshold value in a neural network

2 Answers2