Use this tag for programming-related questions about the softmax function, also known as the normalized exponential function. Questions specific to a certain programming language should also be tagged with that language.
Questions tagged [softmax]
534 questions
3
votes
1 answer
What is the purpose of the dim parameter in torch.nn.Softmax
I don't understand to what does the dim parameter applies in torch.nn.Softmax. There is a warning that tells me to use it and I set it to 1, but I don't understand what I am setting. Where is it being used in the…
gruszczy
- 40,948
- 31
- 128
- 181
3
votes
1 answer
How do you implement softmax with Tensorflow.JS
Using Tensorflow.JS, I am trying to get a machine learning model running with a last dense layer using a softmax activation function. When I try to run it, I receive:
Error when checking target: expected dense_Dense2 to have shape [,1], but got…
blainedwards8
- 57
- 1
- 7
3
votes
2 answers
MultiLabel Soft Margin Loss in PyTorch
I want to implement a classifier which can have 1 of 10 possible classes. I am trying to use the MultiClass Softmax Loss Function to do this. Going through the documentation I'm not clear with what input is required for the function.
The…
Vivek
- 165
- 3
- 9
3
votes
0 answers
How to mask logits for tf.softmax_cross_entropy_with_logits to implement valid actions
I want to compute the softmax_cross_entropy_with_logits of a batch tensor.
I have a batch of logits tensor as input, however I want to mask this tensor before with a boolean mask. The boolean mask is also a batch of masks, in every mask there can be…
superfuzzy
- 346
- 2
- 13
3
votes
1 answer
how can i take the derivative of the softmax output in back-prop
So I am new to ML and trying to make a simple "library" so I can learn more about neural networks.
My question:
According to my understanding I have to take the derivative of each layer according to their activation function so I can calculate their…
Ahmet Aslantaş
- 43
- 1
- 3
3
votes
1 answer
tf.gather runs out of bound, while using a custom softmax_loss function, even though it shouldn't
I'm using a small custom function inside of tf.contrib.seq2seq.sequence_loss(softmax_loss_function=[...]) as a custom sofmax_loss_function:
def reduced_softmax_loss(self, labels, logits):
top_logits, indices = tf.nn.top_k(logits,…
JtheB
- 98
- 6
3
votes
1 answer
Derivative of softmax function in Python
Below is the softmax activation function for a neural network. What is the derivative of this function?
def softmax(z):
e = np.exp(z)
return e / np.sum(e, axis=1)
Krutika Parekh
- 51
- 1
- 1
- 5
3
votes
1 answer
Pytorch softmax along different masks without for loop
Say I have a vector a , with an index vector b of the same length. The indexs are in range 0~N-1, corresponding to N groups. How can I do softmax for every group without for loop?
I'm doing some sort of attention operation here. The numbers for…
Zhang Yu
- 559
- 6
- 15
3
votes
1 answer
Deep Q-Learning : torch.nn.functional.softmax crash
I am following a tutorial, and the function softmax crashes when I use it.
newSignals = [0.5, 0., 0., -0.7911, 0.7911]
newState = torch.Tensor(newSignals).float().unsqueeze(0)
probs = F.softmax(self.model(newState), dim=1)
self.model is a neural…
ThunderTecke
- 31
- 2
3
votes
0 answers
Softmax neural net works with error in implementation, does not work with correct implementation
I have been trying to fix this problem for several days, with no luck. I have been implementing a simple neural net with a single hidden layer from scratch, just for my own understanding. I have successfully implemented it with sigmoid, tanh and…
KOB
- 4,084
- 9
- 44
- 88
3
votes
3 answers
Tensorflow tf.nn.softmax() function performs much better than hand-written softmax
I'm writing a simple logistic regression with tensorflow. I found out that when using tf.nn.softmax, the algorithm converges much quicker, and in the end the accuracy is higher.
If switched to my own implementation of softmax, the network converges…
menphix
- 309
- 4
- 12
3
votes
1 answer
Neural Network using Softmax with strange outputs
I'm trying to build a tensorflow neural network using a sigmoid activation hidden layer and a softmax output layer with 3 classes. The outputs are mostly very bad and I believe it is because I am making a mistake in my model construction because…
Wayde Herman
- 71
- 5
3
votes
1 answer
Could not determine shape of numpy array in a loop containing transpose operation
I have been trying to create a small neural network to learn softmax function with an article from the following website: https://mlxai.github.io/2017/01/09/implementing-softmax-classifier-with-vectorized-operations.html
It works well for a single…
Subhanandh
- 141
- 1
- 11
3
votes
0 answers
Why does TensorFlow's sampled softmax loss stop gradients on sampled classes?
The backbone of TensorFlow's sampled loss functions nce_loss and sampled_softmax_loss is a helper function in tensorflow.python.ops.nn_impl called _compute_sampled_logits.
_compute_sampled_logits takes as input:
weights and biases of the final…
marko
- 230
- 3
- 7
3
votes
1 answer
How scikit learn implements the output layer
In scikit learn, how many neurons are in the output layer? As stated here, you can only specify the hidden layer size and their neurons but nothing about the output layer, thus I am not sure how scikit learn implements the output layer.
Does it…
Medo
- 952
- 3
- 11
- 22