Activation function is a non-linear transformation, usually applied in neural networks to the output of the linear or convolutional layer. Common activation functions: sigmoid, tanh, ReLU, etc.
Questions tagged [activation-function]
343 questions
0
votes
1 answer
Create custom activation function in keras
I want to write a custom activation function with keras.backend for last dense of LSTM like this:
def customactivation(x):
if x <= 0.5:
return 0
else :
return 1
model.add(Dense(1, activation=customactivation))
what should…
Sheida
- 1
0
votes
0 answers
What are examples of non linear activation functions that are in the range of plus to minus infinity?
I am training an auto encoder to reconstruct a TFIDF vectors whose elements can lie in the range of - infinity to + infinity. What apart from linear activation can I use as a possible non linear activation function on the auto encoder's output…
Nasheed Yasin
- 362
- 3
- 11
0
votes
1 answer
how to get what type of activation is used?
I have found that model.layers[index].output prints the info I need. But, I couldn't get what activation function was used by looking at this output:
Tensor("dense_11_1/clip_by_value:0", shape=(?, 256), dtype=float32)
Usually, it is like…
bit_scientist
- 1,496
- 2
- 15
- 34
0
votes
3 answers
how to implement tent activation function in pytorch?
I need to create tent activation function as it is shown in the following image.
I was wondering if there is torch built-in tent activation function?
If not, is there any way to create this activation function?
Thanks
user1538653
- 121
- 1
- 1
- 10
0
votes
1 answer
Understanding choice of loss and activation in deep autoencoder?
I am following this keras tutorial to create an autoencoder using the MNIST dataset. Here is the tutorial: https://blog.keras.io/building-autoencoders-in-keras.html.
However, I am confused with the choice of activation and loss for the simple…
Jane Sully
- 3,137
- 10
- 48
- 87
0
votes
1 answer
Changing Slope of Sigmoid Activation Function in Neural Network - MATLAB
I am currently trying to determine a way to adjust the slope of a sigmoid activation function within a neural network in MATLAB, by some scalar value. For example, as opposed to using tanh(x) I would be using tanh(3x) as my activation function (if…
Jim
- 11
- 4
0
votes
2 answers
Determining Total Number of Parameters Within a Neural Network
If we have an neural network of an input layer with 20 nodes + 2 hidden layers (size 10 and 5), how can we compute the total number of parameters in such a network?
Moreover, how can we know which activation functions will be appropriate for such a…
x89
- 2,798
- 5
- 46
- 110
0
votes
0 answers
Bad results in Autoencoder when using non-linear activation functions in combination with softmax
I am training an autoencoder on data where each observation is a p=[p_1, p_2,...,p_n] where 0
Rutger Mauritz
- 153
- 1
- 12
0
votes
1 answer
How to use different activations in output layer in Keras in R
I want to combine more types of activations in output layer in Keras interface for R. Also, I want to use different loss functions for different outputs. Lets say I want to have first two neurons linear with MSE loss, second 2 neurons sigmoid with…
pikachu
- 690
- 1
- 6
- 17
0
votes
1 answer
Why output layer of sigmoid function gets value 0 or 1 {0,1} instead of getting value in [0,1]
I'm implementing deep deterministic policy gradient (DDPG) to solve my problem by following this tutorial (https://www.youtube.com/watch?v=GJJc1t0rtSU) and using this python source code…
kivegun
- 1
- 1
0
votes
1 answer
Activation function for tensorflow DNNCalssifier's hidden layers
It's possible to customize output layer's activation function for tensorflow's DNNClassifier canned estimator but no documentation about hidden layer's default activation function... Is it a ReLu one ? Is it customizable ?
GerardL
- 81
- 7
0
votes
1 answer
Why sigmoid layer gives worse result than tanh layer in 0-1 regression task?
I'm working with regression to predict an array with 0-1 value (array of bit). The neural network specification is the following (MATLAB):
layers = [
imageInputLayer([1 16 2],'Normalization','none')
fullyConnectedLayer(512)
…
Le Duong Tuan Anh
- 360
- 3
- 12
0
votes
1 answer
Tensorflow.js: Tensor disposed error when using tanh or sigmoid, but not relu activation function
In TensorFlow.js, I created a sequential neural network with 3 dense layers that works when I set the activation function to 'relu', but when I try 'tanh' or 'sigmoid' it throws the error, "Error: Tensor is disposed".
I did a model summary to…
Brett L
- 105
- 1
- 11
0
votes
1 answer
How to print the activation output of specified layers
I am quite new to Keras and deep learning and I have been wanting to print outputs of a section of my layers (named output[x])
Down below you can find a section of the architecture. Do note that I have not provided any reproducible code.
The goal is…
Andrie
- 175
- 1
- 12
0
votes
3 answers
Activation function to get day of week
I'm writing a program to predict when will something happens. I don't know which activation function to get output in day of week (1-7).
I tried sigmoid function but i need to input the predicted day and it output probability of it, I don't want it…
Christophorus Reyhan
- 1,109
- 1
- 7
- 14