2

I am trying to repeat a study (https://www.sciencedirect.com/science/article/pii/S0957417410011711)

In the study they use two different functions, one for the hidden layer and one for the output. On page 5314 they write "A tangent sigmoid transfer function was selected on the hidden layer. On the other hand, a logistic sigmoid transfer function was used on the output layer."

I'm using package "neuralnet" in R.

In order to have a tangent sigmoid transfer function for the hidden layer I can use the code:

act.fct = 'tanh'

But this will create a problem that I will either A) have the SAME function for the output layer.

Or B) I use linear.output = T Which gives me a linear output, but not a sigmoid-function. Is there any possible way for me to have another function for the output layer?

Likewise: If I use act.fct = 'logistic' I will get a logistic sigmoid transfer function throughout the entire network, giving me the correct function for the output layer, but wrong for the hidden layers. Which again only take me halfway.

I have a crude alternative for solving this, a method I'd prefer to not use, It should be possible to use err.fct = and create a customized error function that uses the linear output and runs it through the desired sigmoid function, for the output. Then I run the output from compute command through the sigmoid function, separately. But that seems like a hassle and likely I will mess up somewhere along the way. Any proper/better solution for this?

Patrik Per Lilja
  • 21
  • 2
  • 1
    Which package are you using to specify the neural network? Most NN libraries will let you specify an activation function parameter (like `act.fct` in your case) for **each** layer of the network separately. – hdkrgr Dec 10 '18 at 15:47
  • @hdkrgr using package "neuralnet" for R. Edited question for clarification. – Patrik Per Lilja Dec 10 '18 at 21:20

1 Answers1

2

It doesn't seem like the R package neuralnet supports activation functions in the hidden layers. Check out package keras to solve this for you.

model <- keras_model_sequential() 
model %>% 
  layer_dense(units = 100, activation = 'tanh') %>% 
  layer_dropout(rate = 0.2) %>% 
  layer_dense(units = 1, activation = 'sigmoid')
Corey Levinson
  • 1,553
  • 17
  • 25
  • Thanks, I've switched to Keras now because it seems to be better overall. If anyone sees this and wants to try, my solution suggested in the original question worked. As in, I included it in the error-function. Not a pretty solution, but worked fine. – Patrik Per Lilja Dec 25 '18 at 17:17