Neural Network Approximation Function

Question

I'm trying to test the efficiency of the Neural Networks as approximation functions.

The function I need to approximate has 5 inputs and 1 output, which structure should I use?

I have no idea on what criteria should be applied in order to decide the number of Hidden Layer and the number of Nodes for each layer.

Thank you in advance, Regards

Giuseppe.

Answer 1

I always use a single hidden layer. Theoretically, there are no functions which can be approximated by 2 or more hidden layers that cannot be approximated with one. To make a single hidden layer more complex, add more hidden nodes.

Typically, the number of hidden nodes is varied to observe the effect on model performance (as measured by accuracy or whatever). Too few hidden nodes results in a worse fit due to underfitting (the neural network's output function is too simple, and misses important details in the data). Too many hidden nodes results in a worse fit due to overfitting (the neural network becomes so flexible that it chases every bit of noise in the data).

Answer 2

Note that for classification problems you need at least 2 hidden layers if you want to separate concave polygons.

I'm not sure how the number of hidden layers affects function approximation.