1

Are the bias terms added by default when creating the tensorflow neural network models?

To rephrase, if x is the input to a particular layer, y is the ouput, W is the weight matrix and b is the bias , then the output of the layer is given by, 

  y = W^t x + b

So is the bias added by default when we create the model ?

Uzair Ahmed
  • 119
  • 1
  • 13
  • there are many layers. Which layer are you talking about? – Salvador Dali Jul 01 '17 at 00:54
  • I am talking about convolutional layers in the slim library. My layer looks like this: net = slim.conv2d(net,number_filters,[size_filters,size_filters],stride = stride,scope=scope,padding=padding) .Is the bias already added in the output of this layer? – Uzair Ahmed Jul 02 '17 at 06:37

3 Answers3

1

If you are creating your own model from scratch, you have to create your own trainable variables for the weights and biases explicitly. Tensorflow does not create them by default.

x = tf.placeholder(tf.float32, shape=(None, n))

W = tf.Variable(tf.random_normal([n,m], stddev=0.01))
b = tf.Variable(tf.zeros([m]))

y = tf.matmul(x,W) + b
Joshua Lim
  • 315
  • 3
  • 9
  • I am creating a CNN using the slim library from tensorflow. My first convolutional layer looks like this: net = slim.conv2d(input,number_filters,[size_filters,size_filters], stride=stride,scope=scope,padding=padding) This layer is followed my multiple conv2d layers with max pooling layers as well. I want to know if the biases are already added in the output of this layer. Sorry if the question was not clear initially. – Uzair Ahmed Jul 02 '17 at 06:31
  • In that case, I see that slim does help to simplify the operation. Based on their explanation [here](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/slim#layers). So slim.conv2d already helps you with the creation of the weights and biases – Joshua Lim Jul 02 '17 at 06:52
0

No, You need to define biases and all the variable by yourself. If you not defined biases then your model will be 

y = w^tx
Azad
  • 71
  • 4
  • What if I am using pre-defined layers in from the slim library something like this: et = slim.conv2d(input,number_filters,[size_filters,size_filters]‌​, stride=stride,scope=scope,padding=padding). I am creating a CNN with conv2d and max pooling layers from the slim library in tensorflow. Is the bias already added in the output of this layer? – Uzair Ahmed Jul 02 '17 at 06:38
0

I am not clear which case did you refer in the three possibilities(at least) bellow:

  1. suppose the layer is a basic rnn layer(though you provided only one input and one output), then the answer is yes. Check out this source code. The third parameter of the linear function is true, indicating the bias is added in default.

  2. suppose that you did a linear projection, then you can custom it yourself. A bias can be omitted, for example in this.

  3. and using this API: tf.nn.xw_plus_b, you can define your own bias.

Lerner Zhang
  • 6,184
  • 2
  • 49
  • 66
  • I am creating a CNN using the slim library. I have multiple conv2d layers and the maxpooling layers in my network. I want to know if the bias is already added in the output of the CNN layers. – Uzair Ahmed Jul 02 '17 at 06:39
  • I am not sure about slim library. But I guess internally biases are there as I have seen in tensor flow case when we define RNN layer weight and biases are internally created. – Azad Jul 02 '17 at 12:40
  • Ja, das denke ich auch. – Lerner Zhang Jul 03 '17 at 09:24