0

I'm trying to fuse two networks together into a final pipeline network, but the final model shows the pipeline model as a single Sequential layer rather than its individual layers (see image).

Model representation

Using model.summary() gives me the same result also:

Model: "model"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
conv_lst_m2d_input (InputLayer) [(None, 1000, 216, 1 0                                            
__________________________________________________________________________________________________
...            
__________________________________________________________________________________________________
max_pooling2d_1 (MaxPooling2D)  (None, 44, 1162, 1)  0           conv_lst_m2d_1[0][0]             
__________________________________________________________________________________________________
concatenate (Concatenate)       (None, 44, 1162, 2)  0           max_pooling2d[0][0]              
                                                                 max_pooling2d_1[0][0]            
__________________________________________________________________________________________________
sequential_2 (Sequential)       (None, 216, 1162, 12 5000534     concatenate[0][0]                
==================================================================================================

Here's the code I'm using to merge the networks together:

def fuse_model(output_channels, lrval=0.0001):
    cnn1_mel = cnn_mls(output_channels, lrval=lrval)
    cnn1_sslm = cnn_sslm(output_channels, lrval=lrval)
    combined = keras.layers.concatenate([cnn1_mel.output, cnn1_sslm.output])
    cnn2_in = cnn2(output_channels, lrval=lrval)(combined)
    opt = keras.optimizers.Adam(lr=lrval)  # learning rate
    model = keras.models.Model(inputs=[cnn1_mel.input, cnn1_sslm.input], outputs=[cnn2_in])
    model.compile(loss='categorical_crossentropy', optimizer=opt, metrics=['accuracy'])

    model.summary()
    if not os.path.isfile('Model_Diagram.png'):
        plot_model(model, to_file='Model_Diagram.png', show_shapes=True, show_layer_names=True, expand_nested=True)

    return model

and here's the code of the pipeline model itself:

def cnn2(output_channels, lrval=0.0001):
    model = tf.keras.Sequential()
    model.add(layers.Conv2D(filters=(output_channels * 2),
                            kernel_size=(3, 5), strides=(1, 1),
                            padding='same',  # ((3 - 1) // 2, (5 - 1) * 3 // 2),
                            dilation_rate=(1, 3),
                            activation=layers.LeakyReLU(alpha=lrval), input_shape=(216, 1162, 2)  # (out_chnls,)
                            ))

    model.add(layers.SpatialDropout2D(rate=0.5))
    model.add(
        layers.Conv2D(output_channels * 152, 128, (1, 1), activation=layers.LeakyReLU(alpha=lrval), padding='same'))

    model.add(layers.SpatialDropout2D(rate=0.5))
    model.add(layers.Conv2D(128, 1, (1, 1), padding='same'))

    return model

If it helps, here's the full code: https://gist.github.com/danielathome19/f02fff7f95bf241dddcfe424de87087a

Daniel S.
  • 148
  • 2
  • 9
  • You can use `get_layer` to retrieve a layer summary as `model.get_layer(name=sequential_2).summary()`. For more information you can refer [this](https://stackoverflow.com/a/64415859/14290681) answer. Thanks! –  Sep 08 '21 at 02:18
  • @TFer2 I've been trying to figure this out for weeks now. Thank you so much, every other solution I found was absurdly complex and this gives me exactly what I need! – Daniel S. Sep 09 '21 at 01:36

1 Answers1

1

Thanks to @TFer2 in the comments.

model.get_layer(name='...') 

This was the exact function I needed.

Jeremy Caney
  • 7,102
  • 69
  • 48
  • 77
Daniel S.
  • 148
  • 2
  • 9