what is the default kernel_initializer in keras

Question

In the user manual, it shows the different kernel_initializer below

the main purpose is to initialize the weight matrix in the neural network.

Anyone knows what the default initializer is? the document didn't show the default.

score 118 · Accepted Answer · edited Dec 16 '19 at 07:34

118

Usually, it's glorot_uniform by default. Different layer types might have different default kernel_initializer. When in doubt, just look in the source code. For example, for Dense layer:

class Dense(Layer):
...
    def __init__(self, units,
                 activation=None,
                 use_bias=True,
                 kernel_initializer='glorot_uniform',
                 bias_initializer='zeros',
                 kernel_regularizer=None,
                 bias_regularizer=None,
                 activity_regularizer=None,
                 kernel_constraint=None,
                 bias_constraint=None,
                 **kwargs):

edited Dec 16 '19 at 07:34

mithunpaul

3,268
22
19

answered Oct 23 '17 at 07:35

Sergey Kovalev

9,110
2
28
32

53

+1 you give a man his answer he solves one problem, you teach a man to read source code, he solves problems everyday. – Arka Mallick Jul 25 '18 at 08:16
There is also `glorot_normal`, but I am reading that uniform is preferred for both glorot and he. – Kermit Jan 23 '20 at 18:22

score 6 · Answer 2 · answered Jun 18 '20 at 08:15

6

GlorotUniform, keras uses Glorot initialization with a uniform distribution.r = √(3/fan_avg)

fan_avg = (fan_in + fan_out) /2

number of inputs = fan_in

number of nurons in a layer = fan_out

answered Jun 18 '20 at 08:15

RaaHul Dutta

105
1
4

what is the default kernel_initializer in keras

2 Answers2

Linked

Related