What type of neural network is this?

Question

I'm new to deep learning, (particularly deeplearning4j) and am trying out the examples. Particularly, I want to know which type neural-network is used in the following CSV Example. Is this a deep learning neural network or just "regular neural network". I do understand that difference between normal neural network and deeplearning neural network is that DL algorithms tackle "vanishing gradient" problem, whereas normal neural network don't. I'm bit confused here. What I feel is that following is regular neural network, but I want to confirm.

    MultiLayerConfiguration conf = new NeuralNetConfiguration.Builder()
        .seed(seed)
        .iterations(iterations)
        .activation(Activation.TANH)
        .weightInit(WeightInit.XAVIER)
        .learningRate(0.1)
        .regularization(true).l2(1e-4)
        .list()
        .layer(0, new DenseLayer.Builder().nIn(numInputs).nOut(3)
            .build())
        .layer(1, new DenseLayer.Builder().nIn(3).nOut(3)
            .build())
        .layer(2, new OutputLayer.Builder(LossFunctions.LossFunction.NEGATIVELOGLIKELIHOOD)
            .activation(Activation.SOFTMAX)
            .nIn(3).nOut(outputNum).build())
        .backprop(true).pretrain(false)
        .build();

    //run the model
    MultiLayerNetwork model = new MultiLayerNetwork(conf);
    model.init();
    model.setListeners(new ScoreIterationListener(100));

model.fit(trainingData);

The code - https://github.com/deeplearning4j/dl4j-examples/blob/master/dl4j-examples/src/main/java/org/deeplearning4j/examples/dataExamples/CSVExample.java

score 2 · Accepted Answer · answered Jan 27 '17 at 16:07

That is indeed a regular feedforward neural network with one hidden layer. (If I am reading the code correctly)

The primary concern as to if something is is deep is the number of hidden layers. 0-1 hidden layer is never considered deep. 2 normally isn't. 3+ Normally is.

Whether or not special deep learning methods (Eg ConvNet, DBN Pretraining, or ReLU) are used does not change wether or not the network is deep, but may help with getting better results with it.

As an aside, sometimes other things that are related to getting good repressentations are bunddled in with deep learning,Eg why Skip-gram's from word2vec. Even though they are incredibly shallow.

What type of neural network is this?

1 Answers1