Activation function is a non-linear transformation, usually applied in neural networks to the output of the linear or convolutional layer. Common activation functions: sigmoid, tanh, ReLU, etc.
Questions tagged [activation-function]
343 questions
0
votes
1 answer
Why the computing efficiency of torch.tanh is much higher than the direct expression?
The two ways of computing 'tanh' are shown as follows. Why the computing efficiency of torch.tanh(1) is much higher than the direct expression(2)? I am confused. And where can I find the original code of torch.tanh in pytorch? Dose it written by…
BONNED
- 23
- 1
- 5
0
votes
1 answer
Wordpress dbDelta tables not being created
I'm trying to execute the following code via register_activation_hook. But when I activate the plugin, only the first table is created.
function fsec_create_tables(){
global $wpdb;
$charset_collate = $wpdb->get_charset_collate();
$sql =…
Ross Crawford
- 33
- 4
0
votes
1 answer
AttributeError: 'Tensor' object has no attribute 'numpy' in zReLU
Here is the code which I want to use in my code but I am getting error ( AttributeError: Exception encountered when calling layer "zReLU1" (type zReLU).
) which I attached in a picture!!! Can anyone help?
def get_angle(self, x):
real =…
Fahmida Afrin
- 1
- 2
0
votes
1 answer
Using Prelu in Tensorflow
I am building a reinforcement learning model.
I am trying to use PRelu in my 2D Conv model using tensorflow.
Here is the code for Actor Model.
code:
from tensorflow.keras.layers import Conv2D, Input, MaxPool1D, concatenate, Lambda, Dense,…
AB Music Box
- 81
- 1
- 6
0
votes
1 answer
Custom activation function in Tensorflow with trainable params
I am trying to implement a custom version of the PElu activation function in tensorflow. The custom thing about this activation is the knee of the relu is smoothed. I got the equation from this paper.
Here is the code:
from keras import backend as…
AB Music Box
- 81
- 1
- 6
0
votes
0 answers
Replacing "ReLU" with "ReLU6" for Guided GradCAM visualization
I am trying to implement Guided GradCAM to understand my custom MobileNetV2 model predictions.
In the following block of code, is it fine to replace 'Relu' with 'Relu6' (since authors of MobilenetV2 have used Relu6 as the activation)?
Asking this…
seetha92
- 1
0
votes
1 answer
ValueError: Exception encountered when calling layer "p_re_lu_1" (type PReLU)
I have been facing following error while iterating through a a range of neurons and activation functions. The error is encoutered only in case of Prelu and none of the other activation functions result in the same affect. Neither am I able to deduce…
Shobhit Kulshreshtha
- 11
- 5
0
votes
1 answer
Custom activation function dependant on other output nodes in Keras
I would like to predict a multi-dimensional array using Long Short-Term Memory (LSTM) networks while imposing restrictions on the shape of the surface of interest.
I thought to accomplish this by setting some elements of the output (regions of the…
Mr Frog
- 296
- 2
- 16
0
votes
1 answer
Different activation function based on input
I am trying to build a Keras neural network where the activation function at the output layer (conditionally) depends on the inputs. The activation function is quite complicated, so as a simpler example, consider the following:
def…
Philip Winchester
- 13
- 4
0
votes
0 answers
Performance comparison of different activation functions
I was trying to find a way to compare the test accuracy and test loss of different activation functions (such as tanh, sigmoid, relu), so I came up with this script:
import numpy as np
from sklearn import datasets
from sklearn.model_selection import…
Amirhossein Rezaei
- 297
- 3
- 14
0
votes
1 answer
What activation layers learn?
I am trying to figure out what CNN architecture after every activation layers. Therefore, I have written a code to visualize some activation layers in my model. I used LeakyReLU as my activation layer. This is the figureLeakyRelu after Conv2d +…
killermama98
- 45
- 5
0
votes
0 answers
Error when running the model with custom square activation function
An error occur when try to execute a custom activation function all the commands work until reaching the last one hits an error!
Tensorflow version is: 2.9.1
keras version is: 2.9.0
Thanks in advance.
The code
import tensorflow
from…
Anmar Ali
- 1
- 2
0
votes
0 answers
Testing Concept Activation Vector (TCAV) for Pytorch
I am trying to compute the Testing Concept Activation Vectors (TCAV, as described here) vectors for different concepts for my classification model. So far, I haven't successfully found code online for Pytorch models so I have decided to rewrite it…
rkraaijveld
- 23
- 4
0
votes
0 answers
How to create PSS and SS activation functions in Keras
I'm using keras with tensorflow backend. My goal is to add my custom activation functions (PSS, SS) but I don't know how can i implement them.
Positive Smooth Staircase (PSS) activation function
Where n is number of output labels, w is a…
0
votes
2 answers
Question about Perceptron activation threshold
I have a Perceptron written in Javascript that works fine, code below. My question is about the threshold in the activation function. Other code I have seen has something like if (sum > 0) {return 1} else {return 0}. My perceptron only works with if…
Mike
- 3
- 3