4

I couldn't find a clear consensus on this.

What does 2-way softmax mean, and how is it different from n-way softmax?

The definition is given by Geoffrey Hinton in his Coursera course Neural Networks for Machine Learning in Quiz 4 to be:

a softmax unit with 2 elements

I'm completely new to this, so I don't understand whether it means:

  1. There are only two inputs into the neuron which computes the softmax?
  2. The output of the function is a vector with 2 elements? E.g. [0.5, 0.2]
  3. The output can only be 0 or 1?

Any clarification is appreciated.

Andriy Drozdyuk
  • 58,435
  • 50
  • 171
  • 272

1 Answers1

-1

As far as I understood the question: 2-way Softmax means, the Neural Network should decide, weather it is this (x)or that. So it can only output a binary decision.

N-way means, that you have multiple output. For instance: Your network should decide, if a given picture shows a dog or a cat (2-way)

Another Network should tell, if the picture shows a car, a ball, a house or a human, etc... (n-way). Check this out: https://github.com/Kulbear/deep-learning-nano-foundation/wiki/ReLU-and-Softmax-Activation-Functions

Technically spoken: A bunch of neurons in a network are connected with mathematical operations, weights and biases...

Most models I use, use numbers between 0 and 1 (double/float). But in classification networks, you want to have one defined result. For the cat/dog network, you would have 2 output neurons (one for the cat-probability and another one for the dog-probability). So a given picture with a clear fat dog will produce something like (0.265, 0.995). So its most probably a dog. Applying the 2-way softmax here, will result in ~(0.2, 0.8). So "its a dog"

Basically: It normalizes the fuzzy output to enable the user to make a clear descision. Read the link provided. Its good...

Sauer
  • 1,429
  • 4
  • 17
  • 32
  • Sorry, how will you get (0,1) if softmax itself also outputs probabilities? I mean in a softmax group, each neuron will output some probability, which all together will sum up to one... How is the number "1" or "0" produced? – Andriy Drozdyuk Nov 06 '17 at 22:46
  • Ok, I just saw "4-way" softmax, to classify four objects. So it seems to me that 4-way softmax is a softmax group with four elements. – Andriy Drozdyuk Nov 06 '17 at 23:07