-1

I am using feed forward, gradient descent backpropagation neural networks.

Currently I have only worked with non-linear networks where tanh is activation function.

I was wondering. What kind of tasks would you give to a neural networks with non-linear activation function and what kind of tasks for linear?

I know that network with linear activation function are used to solve linear problems. What are those linear problems? Any examples?

Thanks!

Gyiove Sparkle
  • 23
  • 3

1 Answers1

0

I'd say never, since composition of linear functions is still linear using a neural network with linear activations is just a way to complicate linear regression.

Whether to choose a linear model or something more complicated is up to you and depends on the data you have; this is (one of the reasons) why it is customary hold out some data during training and use it to validate the model. Other ways of testing models are residuals analysis, hypothesis testing, and so on

BlackBear
  • 22,411
  • 10
  • 48
  • 86
  • I am using neural networks for pattern recognition. The problem is that there is a pattern and then there are hints of weither pattern might accrue, for example one input is being week day (Monday etc..), other one a year month ( January etc... ) and then there are 9 inputs for data itself where patter might be hidding and then the output telling what is the probability that It's that specific pattern i trained neural network to recognition. – Gyiove Sparkle Apr 07 '16 at 16:43
  • Since you want a probability use sigmoid activations at least in the output layer (which should have only one neuron). If the pattern is time-dependent you should probably switch to a LSTM network or at least provide multiple "snapshots" (present and some months in the past) as input to the feedforward network – BlackBear Apr 07 '16 at 17:02