0

I am making a AutoEncoder on Tensorflow which takes input as a 3 D Matrix whose value lie in the range of [-1,1]. What is the optimal activation function for this scenario?

Also, what is the rule of thumb in selecting the activation function w.r.t to the input ranges?

a_parida
  • 606
  • 1
  • 7
  • 26

2 Answers2

0

First of all, it is generally advisable to start the network with batch normalization, which would more or less confine the values between -1 and 1 anyway.

The activation function of the hidden layers should have non-linearity to be able handle higher levels of complexity. So I'd choose relu or one of its variants.

At the end of the network, at the last layer, you can have a dense layer with no activation function to allow for mapping the negative values.

Peter Szoldan
  • 4,792
  • 1
  • 14
  • 24
0

There is no "Rule of thumb" when it comes to picking an activation function.

I use this guide to pick my activation function depending on the complexity

Also refer to this answer for some more guidance

Siddharth Chabra
  • 448
  • 6
  • 22