0

I need to write my CNN model as a Theano function with my weights already set by Keras (Tensorflow as the backend), but I am unsure about how to add the bias values associated with each layer.

This solution How can I get a 1D convolution in theano works nicely to write a single layer as a Theano function, but I need to stack my weights with the biases from each layer

Simplified version of my code:

model = Sequential([

    InputLayer(batch_input_shape=(None,100,1)),
    Convolution1D(nb_filter=16, filter_length=8, activation='relu', border_mode='same', init='he_normal', input_shape=(None,100,1)),
    Convolution1D(nb_filter=32, filter_length=8, activation='relu', border_mode='same', init='he_normal'), 
    MaxPooling1D(pool_length=4),
    Flatten(),
    Dense(output_dim=32, activation='relu', init='he_normal'),
    Dense(output_dim=1, input_dim=32, activation='linear'),
])

How do you add the bias weights to the CNN layer?

For instance, the weights of my first layer have the dimensions: (8, 1, 1, 16)

With a bias with dimensions: (16,)

Which is easy enough to concatenate together to get dimensions: (9, 1, 1, 16)

but for the next layer I have dimensions: (8, 1, 16, 32)

with a bias with dimensions: (32,)

How can I combine this into one weight matrix? To put into the Theano T.signal.conv.conv2d function?

Community
  • 1
  • 1
Starnetter
  • 837
  • 2
  • 8
  • 21
  • Not sure what you mean. Usually convolutional networks have a single bias for each feature map. What are you trying to accomplish? Do you want to build your network and then change your bias to a desired value? – maz Mar 18 '17 at 04:32
  • No, I have my weights set already and need to write them in the form of a function similar to this http://stackoverflow.com/questions/42792405/return-inverse-hessian-matrix-at-the-end-of-dnn-training-and-partial-derivatives but for a CNN. But I am unsure of how the CNN layers and their biases are combined. – Starnetter Mar 18 '17 at 07:13
  • I still do not understand what you mean. If you want to add biases to a convolutional layer you could simply pass the argument bias=True (keras 1 ) or pass use_bias=True (keras 2) to your convolutional layer. – maz Mar 18 '17 at 10:02
  • I already have all of my weights set, including the biases. I am trying to write my model in the form of a function like I did in this solution stackoverflow.com/questions/42792405/… but for Dense layers it is fairly straightforward to combine the layers with the corresponding biases. I am just having a hard time understanding how the layer weights and bias weights are stacked for a CNN layer. How do you stack weights with dimensions (8, 1, 16, 32) with biases with dimensions (32,)? I need to write it in the form of a Theano function to return the Jacobian matrix for error propagation – Starnetter Mar 18 '17 at 15:33

0 Answers0