I am training an auto encoder to reconstruct a TFIDF vectors whose elements can lie in the range of - infinity to + infinity. What apart from linear activation can I use as a possible non linear activation function on the auto encoder's output layer? The activation function is desired to give outputs in the + infinity to - infinity range.
All my other NN layers, including the 'Bottleneck' have tanh activation. I am using cosine_proximity loss and Adagrad optimization to guide gradient descent.
I am implementing the above in Keras with Tensorflow backend.
