0

I am constructing a deep neural network from scratch and I want to implement the softmax http://neuralnetworksanddeeplearning.com/chap3.html#softmax distributed function.
I am using breeze for that but it is not working as expected.
The documentation is also poor with very few examples, so it is difficult for me to understand how I should use it.

here is an example : I have an ouput array that contains 10 dimensions.
I have my label array also.

Z contains 10 rows with the weighted values.
My label array contains also 10 rows and one is set to 1 to specify which row is the expected result.

lab(0) = 1
lab(1 to 9) = 0

my code :

def ComputeZ(ActivationFunction : String, z:Array[Double], label:Array[Double]) : Array[Double] = {
    ActivationFunction match {
      case "SoftMax" => **val t = softmax(z,label)**
      t
    }
  }

I was expecting having a distributed probability with a total of 1 for the 10 rows but it returns actually the same values as Z.
I don't know what I am doing wrong thanks for your help

ezl
  • 23
  • 4

1 Answers1

1

Your question seems a little bit confusing to me. I mean, creating a SoftMax from scratch has nothing to do with the label or the real output value. A Softmax function is used to create a valid output probability distribution of a neural network, used in multiclass classification problems. As I see you have a one hot vector as label, it seems that you want to implement a CrossEntropy criterion or some error function that evaluates the divergence of the prediction distribution and the label distribution. That needs the output prediction probability distribution(applying your Softmax to the output layer) and the one hot vector of the output.

I watched the code of the softmax function in breeze but I don´t see a Layer implementation and it doesn´t do what I was expecting. Have in mind that you need a forward an a backward function.

Emiliano Martinez
  • 4,073
  • 2
  • 9
  • 19
  • thanks for your reply. The size of the array is the number of classes. This call comes when the deep neural network reaches the output layer before calculating the loss function. I have to probably write the softmax function by myself. – ezl Jan 11 '23 at 19:10
  • 1
    Take a look to the SoftMax function in BigDL which is implemented in Scala – Emiliano Martinez Jan 12 '23 at 09:01