Keras: Feeding in part of previous layer to next layer, in CNN

Question

I am trying to feed in the individual kernel outputs of the previous layer to a new conv filter, to get the next layer. To do that, I tried passing each of the kernel outputs through a Conv2D, by calling them by their index. The function I used is:

def modification(weights_path=None, classes=2):

    ###########
    ## Input ##
    ###########

    ### 224x224x3 sized RGB Input
    inputs = Input(shape=(224,224,3))

    #################################
    ## Conv2D Layer with 5 kernels ##
    #################################

    k = 5
    x = Conv2D(k, (3,3), data_format='channels_last', padding='same', name='block1_conv1')(inputs)
    y = np.empty(k, dtype=object)
    for i in range(0,k):
        y[i] = Conv2D(1, (3,3), data_format='channels_last', padding='same')(np.asarray([x[i]]))
    y = keras.layers.concatenate([y[i] for i in range (0,k)], axis=3, name='block1_conv1_loc')
    out = Activation('relu')(y)
    print ('Output shape is, ' +str(out.get_shape()))

    ### Maxpooling(2,2) with a stride of (2,2)
    out = MaxPooling2D((2,2), strides=(2,2), data_format='channels_last')(out)

    ############################################
    ## Top layer, with fully connected layers ##
    ############################################

    out = Flatten(name='flatten')(out)
    out = Dense(4096, activation='relu', name='fc1')(out)
    out = Dropout(0.5)(out)
    out = Dense(4096, activation='relu', name='fc2')(out)
    out = Dropout(0.5)(out)
    out = Dense(classes, activation='softmax', name='predictions')(out)

    if weights_path:
        model.load_weights(weights_path)

    model = Model(inputs, out, name='modification')

    return model

But this is not working, and is throwing the following error:

Traceback (most recent call last):
  File "sim-conn-edit.py", line 137, in <module>
    model = modification()
  File "sim-conn-edit.py", line 38, in modification
    y[i] = Conv2D(1, (3,3), data_format='channels_last', padding='same')(np.asarray([x[i]]))
  File "/home/yx96/anaconda2/lib/python2.7/site-packages/keras/engine/topology.py", line 511, in __call__
    self.assert_input_compatibility(inputs)
  File "/home/yx96/anaconda2/lib/python2.7/site-packages/keras/engine/topology.py", line 408, in assert_input_compatibil
ity
    if K.ndim(x) != spec.ndim:
  File "/home/yx96/anaconda2/lib/python2.7/site-packages/keras/backend/tensorflow_backend.py", line 437, in ndim
    dims = x.get_shape()._dims
AttributeError: 'numpy.ndarray' object has no attribute 'get_shape'

I fed in layer x[i] as [ x[i] ] to meet the dimention requirements of a Conv2D layer. Any help with solving this problem will be deeply appreciated!

Have you tried just passing `x[i:i+1]`? --- Curiosity.... what are you trying to do with that model? — Daniel Möller, Jun 28 '17 at 20:19
Well, `x[i:i+1]` will correspond to 2 kernel outputs, right, not 1? Also, the dimension of `x` turns out to be `(None, 5, 224, 224)` instead of `(5, 224, 224)`. About why I am trying it out, I am trying to learn implementation of Conv Nets, and tried exploring different stuff with keras, and got stuck here. — Prabaha, Jun 29 '17 at 17:51
`x[i:i+1]` returns only one element, the element at `i`, but it is returned as an array/tensor instead of a single element. (Anyway, it doesn't work). -- Now, it's normal to be `(None, .....)`. Keras creates the none to represent the "batch_size" (how many examples you have). -- And, if you're learning, I suggest you don't try doing that. Unless you have a very clear reason to do that, it's not something anyone would normally do. Just use one Conv2D layer and it will take care of everything: `out = Conv2D(filters,.....)(x)` — Daniel Möller, Jun 29 '17 at 18:25
I don't think that separating parts of the tensors is something usually done. If you explain with details what you want to do, why do you need to separate parts and how many parts you want, maybe we can find another way of doing it. — Daniel Möller, Jul 06 '17 at 18:07
@Daniel If you give me your email address, I can send you a detailed description of the problem. — Prabaha, Jul 06 '17 at 18:30

score 2 · Accepted Answer · answered Jul 07 '17 at 00:32

After posting this and this questions in StackOverflow, and some personal exploring, I came up with a solution. One can possibly do this with Lambda layers; by calling a Lambda layer to extract a sub part of the previous layer. For example, if the Lambda function is defined as,

def layer_slice(x,i):
    return x[:,:,:,i:i+1]

and then, called as,

k = 5
x = Conv2D(k, (3,3), data_format='channels_last', padding='same', name='block1_conv1')(inputs)
y = np.empty(k, dtype=object)
for i in range(0,k):
    y[i] = Lambda(layer_slice, arguments={'i':i})(x)
    y[i] = Conv2D(1,(3,3), data_format='channels_last', padding='same')(y[i])
y = keras.layers.concatenate([y[i] for i in range (0,k)], axis=3, name='block1_conv1_loc')
out = Activation('relu')(y)
print ('Output shape is, ' +str(out.get_shape()))

it should effectively feed in individual kernel outputs to a new Conv2D layer. The layer shapes and corresponding number of trainable parameters being obtained from model.summary() matches the expectation. Thanks to Daniel for pointing out that Lambda layers cannot have trainable weights.

For those who use "theano" backend, add the `output_shape=(width,heigth,1)` argument in the Lambda layer. — Daniel Möller, Jul 07 '17 at 19:01

score 2 · Answer 2 · answered Jul 07 '17 at 18:36

2

Prabaha. I know you've solved your problem, but now I see your answer, you can do that without using the lambda layer too, just split the first Conv2D in many. One layer with k filters is equivalent to k layers with one filter:

for i in range(0,k):
    y[i] = Conv2D(1, (3,3), ... , name='block1_conv'+str(i))(inputs)     
    y[i] = Conv2D(1,(3,3), ...)(y[i])
y = Concatenate()([y[i] for i in range (0,k)])
out = Activation('relu')(y)

You can count the total parameters in your answer and in this answer to compare.

answered Jul 07 '17 at 18:36

Daniel Möller

84,878
18
192
214

1

This is exactly the implementation I had been using. But for the first stack of `k` filters, I wanted to load pre trained weights, which can be done by the `by_name=True` flag in `load_weights`. By breaking one layer of `k` filters into `k` layers of one filter, I did not have a convenient way of loading the pre-trained weights. That is why, I had to come up with something that kept layer `x` intact. – Prabaha Jul 07 '17 at 18:46

Keras: Feeding in part of previous layer to next layer, in CNN

2 Answers2

Linked