Use this tag for programming-related questions about the softmax function, also known as the normalized exponential function. Questions specific to a certain programming language should also be tagged with that language.
Questions tagged [softmax]
534 questions
7
votes
2 answers
Different Sigmoid Equations and its implementation
When reviewing through the Sigmoid function that is used in Neural Nets, we found this equation from https://en.wikipedia.org/wiki/Softmax_function#Softmax_Normalization:
Different from the standard sigmoid equation:
The first equation on top…
alvas
- 115,346
- 109
- 446
- 738
6
votes
1 answer
Tensorflow issue with softmax
I have a Tensorflow multiclass classifier that is generating nan or inf while computing probabilities using tf.nn.softmax. See the following snippet (logits is of shape batch_size x 6, since I have 6 classes and the output is one-hot encoded).…
Nik
- 5,515
- 14
- 49
- 75
6
votes
1 answer
How to use PyTorch to softmax only the upper triangular elements of a matrix?
Given input like:
tensor([[[1.9392, -1.9266, 0.9664],
[0.0000, -1.9266, 0.9664],
[0.0000, -0.0000, 0.9664]]])
My desired output is:
tensor([[[0.4596, 0.0096, 0.1737],
[0.0000, 0.0096, 0.1737],
[0.0000,…
shu.liu
- 63
- 3
6
votes
2 answers
return the top_k masked softmax of each row for a 2D tensor
For any 2D tensor like
[[2,5,4,7],
[7,5,6,8]],
I want to do softmax for the top k element in each row and then construct a new tensor by replacing all the other elements to 0.
The result should be to get the softmax of top k (here k=2) elements…
clement116
- 317
- 2
- 11
6
votes
1 answer
Pytorch - Pick best probability after softmax layer
I have a logistic regression model using Pytorch 0.4.0, where my input is high-dimensional and my output must be a scalar - 0, 1 or 2.
I'm using a linear layer combined with a softmax layer to return a n x 3 tensor, where each column represents the…
Gustavo Silva
- 315
- 2
- 4
- 11
6
votes
1 answer
What's the difference between Softmax and SoftmaxWithLoss layer in caffe?
While defining prototxt in caffe, I found sometimes we use Softmax as the last layer type, sometimes we use SoftmaxWithLoss, I know the Softmax layer will return the probability the input data belongs to each class, but it seems that SoftmaxwithLoss…
Eric Luo
- 339
- 3
- 11
6
votes
2 answers
how to interpret the "soft" and "max" in the SoftMax regression?
I know the form of the softmax regression, but I am curious about why it has such a name? Or just for some historical reasons?
Shuai Wang
- 75
- 5
5
votes
2 answers
Why is softmax classifier gradient divided by batch size (CS231n)?
Question
In CS231 Computing the Analytic Gradient with Backpropagation which is first implementing a Softmax Classifier, the gradient from (softmax + log loss) is divided by the batch size (number of data being used in a cycle of forward cost…
mon
- 18,789
- 22
- 112
- 205
5
votes
1 answer
input for torch.nn.functional.gumbel_softmax
Say I have a tensor named attn_weights of size [1,a], entries of which indicate the attention weights between the given query and |a| keys. I want to select the largest one using torch.nn.functional.gumbel_softmax.
I find docs about this function…
namespace-Pt
- 1,604
- 1
- 14
- 25
5
votes
2 answers
How to fix “ValueError: not enough values to unpack (expected 2, got 1)”
I am trying to do sentiment analysis on a german tweet-data-set with the bert-base-german-cased modell which i imported over transformers from hugginface.
To be able to calculate the predicted probabilities i want to Softmax of Numpy and here does…
Skalonga
- 75
- 3
- 8
5
votes
1 answer
How tensorflow softmax add an unknown class?
I set up an ocr classification system using Tensorflow.
Here is graph:
def build_graph(top_k):
# with tf.device('/cpu:0'):
keep_prob = tf.placeholder(dtype=tf.float32, shape=[], name='keep_prob')
images = tf.placeholder(dtype=tf.float32,…
Wesley
- 1,857
- 2
- 16
- 30
5
votes
2 answers
PyTorch softmax with dim
Which dimension should softmax be applied to ?
This code :
%reset -f
import torch.nn as nn
import numpy as np
import torch
my_softmax = nn.Softmax(dim=-1)
mu, sigma = 0, 0.1 # mean and standard deviation
train_dataset = []
image = []
image_x =…
blue-sky
- 51,962
- 152
- 427
- 752
5
votes
2 answers
What is the difference between softmax and log-softmax?
The difference between these two functions that has been described in this pytorch post: What is the difference between log_softmax and softmax?
is: exp(x_i) / exp(x).sum()
and log softmax is: log(exp(x_i) / exp(x).sum()).
But for the Pytorch code…
Amogh Mishra
- 1,088
- 1
- 16
- 25
5
votes
4 answers
Keras softmax activation, category_crossentropy loss. But output is not 0, 1
I trained CNN model for just one epoch with very little data. I use Keras 2.05.
Here is the CNN model's (partial) last 2 layers, number_outputs = 201. Training data output is one hot encoded 201 output.
model.add(Dense(200, activation='relu',…
user6101147
- 185
- 1
- 3
- 14
5
votes
5 answers
Softmax function of a numpy array by row
I am trying to apply a softmax function to a numpy array. But I am not getting the desired results. This is the code I have tried:
import numpy as np
x = np.array([[1001,1002],[3,4]])
softmax = np.exp(x - np.max(x))/(np.sum(np.exp(x -…
Pranay Aryal
- 5,208
- 4
- 30
- 41