Use this tag for programming-related questions about the softmax function, also known as the normalized exponential function. Questions specific to a certain programming language should also be tagged with that language.
Questions tagged [softmax]
534 questions
3
votes
1 answer
How are matrices multiplied in Hierarchical Softmax model?
As I understood, the simple word2vec approach uses two matrices like the following:
Assuming that the corpus consists of N words.
Weighted input matrix (WI) with dimensions NxF (F is number of features).
Weighted output matrix (WO) with dimensions…
abbudeh
- 65
- 8
3
votes
2 answers
Why is softmax function necessory? Why not simple normalization?
I am not familiar with deep learning so this might be a beginner question.
In my understanding, softmax function in Multi Layer Perceptrons is in charge of normalization and distributing probability for each class.
If so, why don't we use the simple…
soshi shimada
- 425
- 1
- 7
- 21
3
votes
0 answers
Accessing the softmax output of previous RNN state in Keras
The recurrence formula in my RNN is h(t) = tanh(W.x(t) + U.h(t-1) + V.O(t-1) + b), where O(t-1) is the classification output (Softmax output) of the RNN at time t-1. How can I access this output at the next time step?
I guess I have to write a…
Monaj
- 854
- 9
- 16
3
votes
2 answers
tensorflow - softmax ignore negative labels (just like caffe)
In Caffe, there is an option with its SoftmaxWithLoss function to ignore all negative labels (-1) in computing probabilities, so that only 0 or positive label probabilities add up to 1.
Is there a similar feature with Tensorflow softmax loss?
Link L
- 439
- 7
- 25
3
votes
1 answer
(matlab) MLP with relu and softmax not working with mini-batch SGD and produces similar predictions on MNIST dataset
I implemented a multilayer perceptron with 1 hidden layer on MNIST dataset. The activation function in hidden layer is leaky(0.01) ReLu and output layer has a softmax activation function. The learning method is mini-batch SGD. The network structure…
Ludwig Zhou
- 1,026
- 1
- 11
- 23
3
votes
0 answers
How to feed parameter ''dy" of ‘cudnnSoftmaxBack()' in cuDNN API?
I want to implement to LeNet-5 with cuDNN, and try to train the net on MNIST data set.
The last layer of the net is 'Softmax', and I use the function 'cudnnSoftmaxForward()' in the forward process. And then, I want to use the function…
tuonion
- 31
- 2
3
votes
1 answer
Action selection with softmax?
I know this might be a pretty stupid question to ask, but what the hell..
I at the moment trying to implement soft max action selector, which uses the boltzmann distribution.
Formula
What I am bit unsure about, is how how do known if you want to…
Vato
- 37
- 1
- 8
3
votes
3 answers
Calculate the softmax of an array column-wise using numpy
Following https://classroom.udacity.com/courses/ud730/lessons/6370362152/concepts/63815621490923, I'm trying to write a "softmax" function which, when given a 2-dimensional array as input, calculates the softmax of each column. I wrote the following…
Kurt Peek
- 52,165
- 91
- 301
- 526
3
votes
2 answers
Trying to compute softmax values, getting AttributeError: 'list' object has no attribute 'T'
First off, here is my code:
"""Softmax."""
scores = [3.0, 1.0, 0.2]
import numpy as np
def softmax(x):
"""Compute softmax values for each sets of scores in x."""
num = np.exp(x)
score_len = len(x)
y = [0] * score_len
for…
SDG
- 2,260
- 8
- 35
- 77
3
votes
1 answer
How can I retrieve the output from both fc and softmax layers from Inception-v3 in a single run?
I would like to extract the output of both 'pool_3:0' and 'softmax:0' layers. I could run the model twice and, for each run, extract the output of a single layer, but it's a bit wasteful. Is it possible to do it running the model only once?
I'm…
Yamaneko
- 3,433
- 2
- 38
- 57
3
votes
0 answers
Facial Keypoints Detection using Softmax Regression
I am trying to build a basic Softmax Regression model using Tensorflow for Kaggle Facial Keypoints Detection competition.
I followed Tensorflow beginner MNIST example for Softmax Regression model and Daniel Nouri's blog for data structuring.
The…
turtle
- 1,619
- 1
- 14
- 30
3
votes
3 answers
Neural Network with softmax activation
edit:
A more pointed question:
What is the derivative of softmax to be used in my gradient descent?
This is more or less a research project for a course, and my understanding of NN is very/fairly limited, so please be patient :)
I am currently in…
Cambium
- 19,152
- 3
- 26
- 19
3
votes
1 answer
using softmax in nnet R for target column with more than 2 states
I am using the nnet package for classification of a target column with 3 states
model <- nnet(targetcolumn ~ ., data=DATAFRAME)
But I want it to use entropy instead of default softmax and when I set softmax=false , it fails with the error :
model…
user2912902
- 327
- 1
- 7
- 17
2
votes
1 answer
CrossEntropyLoss showing poor accuracy on 2d output
I'm trying some experiments on a simple neural network that just tries to learn the squares of some random numbers, represented as arrays of decimal digits, code copied below, with changes indicated by comments.
The version using nn.Softmax(dim=2)…
rwallace
- 31,405
- 40
- 123
- 242
2
votes
0 answers
Boltzmann Exploration (softmax) efficient action probabilities update (and roulette wheel action selection)
I have reinforcement learning problem which for this purpose can be substituted by multi-armed bandit. There are various reinforcement learning techniques applicable to this problem, two being:
Epsilon greedy where the best action is selected with…
eXPRESS
- 425
- 2
- 4
- 19