What is the best way to normalize negative/non numerical data for tanh activation function neural networks

Question

I'm using feed forward, gradient descent, backpropagation neural networks where hidden/output neurons are using tanh activation function and input neurons are linear.

What is the best way, in your opinion, for normalizing numerical data if:

1. Maximum number is known and for example maximum positive number would be 1000 and maximum negative -1000.

2. Maximum number is unknown.

And if I should keep the maximum numbers same for all inputs or would it be okay if network's inputs have different normalizing way?

Thanks!

Answer 1

If max and min are known, the easiest normalization is :

normalized = (val - min) / (max - min)

If max is unknown, you can normalize based on the data you do have, with the knowledge that tanh has good characteristics for values that exceed a magnitude of 1.

You should normalize different inputs based on the range of values of those inputs and you may use different normalization procedures for different inputs.