1

Is it possible in Keras to create a Convolutional layer (Conv2D) such that the kernels are initialized according to an initializer (like he_normal, glorot_uniform, etc.), but are created to be identical for every kernel?

In other words, I'd like to initialize one of the kernels using kernel_initializer='he_normal', and then copy and use that initialized weight matrix to initialize all of the other kernels with, in that layer only.

In semi-pseudo-code fashion, this is (similar in concept) to what I'm looking for:

n_filters = 64

x = Conv2D(1, 3,... kernel_initializer='he_normal')

he_normal_kernel = *the kernel weight matrix that was just created*  #copy kernel weight matrix 
he_normal_kernels = he_normal_kernel * n_filters #make n_filters copies of that matrix

x = Conv2D(n_filters, 3,... kernel_initializer=he_normal_kernels) # use those as the initialization of our convolutional layer

I'm impartial to exactly how to implement this, as long as it works as intended.

While we're here, are there any inherent theoretical reasons why this might be a bad idea?

Thanks!

J Doug
  • 71
  • 7
  • This might help: https://stackoverflow.com/questions/50733569/how-to-set-seed-value-of-kernel-initializer-glorot-uniform-in-keras – Luke DeLuccia Jan 24 '19 at 22:53
  • Thanks! @LukeDeLuccia Does setting the value of the seed mean that every initialized matrix in that layer uses that seed and are therefore identical to eachother, OR just that the matrices are somehow initialized in such a way that they are different from each other, but identical to themselves between fresh training sessions? – J Doug Jan 25 '19 at 04:00

0 Answers0