0

I have a Neural Network with two hidden layers. I want to add a bias unit only to the second hidden layer. How do I do that?

The code for my network is as follows:

nn = FeedForwardNetwork()
inLayer = LinearLayer(numFeatures)
hiddenLayer1 = LinearLayer(numFeatures+1)
hiddenLayer2 = SigmoidLayer(numFeatures+1)
outLayer = LinearLayer(1)

nn.addInputModule(inLayer)
nn.addModule(hiddenLayer1)
nn.addModule(hiddenLayer2)
nn.addOutputModule(outLayer)

in_to_hidden1 = FullConnection(inLayer, hiddenLayer1)
hidden1_to_hidden2 = FullConnection(hiddenLayer1, hiddenLayer2)
hidden2_to_out = FullConnection(hiddenLayer2, outLayer)

nn.addConnection(in_to_hidden1)
nn.addConnection(hidden1_to_hidden2)
nn.addConnection(hidden2_to_out)
nn.sortModules()
Adarsh Chavakula
  • 1,509
  • 19
  • 28

1 Answers1

4

This is fairly simple task. First you have to create Bias module:

bias = BiasUnit()

Then add it to your NeuralNetwork, so:

nn = FeedForwadNetwork()
nn.addModule(bias)

Then, assuming you already added other layers you have to connect bias to hidden layer of your choice:

bias_to_hiden = FullConnection(bias, hiden_layer)

And then add it to neural network:

nn.addConnection(bias_to_hiden)

Apart from that you do everything same as before.

For the reference check code of the buildNetwork function from pybrain.tools.shortcuts module. Here is some code that connects bias unit to other layers (lines 75-79):

if opt['bias']:
    # add bias module and connection to out module, if desired
    n.addModule(BiasUnit(name='bias'))
    if opt['outputbias']:
        n.addConnection(FullConnection(n['bias'], n['out']))

Hope that helped.

Pawel Wisniewski
  • 430
  • 4
  • 21
  • Is there also a shortcut to add a bias unit to all the hidden layers (without making an explicit "for" loop :) )? – schmi Jun 28 '16 at 13:40