Merging two custom Keras models into new model

Question

I am trying to build a custom model that feeds the input to two pretrained Unet models which I defined in a separate file (without the Sequential class), locks their weights, concatenates their outputs, and finally end it with a 1x1 2D convolution layer to produce a segmentation map.

So far, I have seen ways to achieve this with pretrained models from the pre-loaded models in the keras.applications API, however, I cannot find a way to finalize this with custom models. When I try to create a DualUnet object, I get the following:

Traceback (most recent call last):

  File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\spyder_kernels\py3compat.py:356 in compat_exec
    exec(code, globals, locals)

  File c:\users\itagl\github\proj\train.py:209
    m1 = DualUnet(FILE_PATH+'/saved_models/Unet/No_Aug.h5',

  File ~\github\proj\models\DualUnet.py:32 in __init__
    concat = tf.keras.layers.concatenate([self.model1.layers[-1], self.model2.layers[-1]])

  File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\keras\layers\merge.py:968 in concatenate
    return Concatenate(axis=axis, **kwargs)(inputs)

  File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\keras\utils\traceback_utils.py:67 in error_handler
    raise e.with_traceback(filtered_tb) from None

  File ~\AppData\Local\Programs\Python\Python310\lib\site-packages\keras\layers\merge.py:500 in build
    if not isinstance(input_shape[0], tuple) or len(input_shape) < 1:

TypeError: 'NoneType' object is not subscriptable

DualUNet

import tensorflow as tf
from tensorflow.keras.models import clone_model

def freeze_all(model):
    
    for i, l in enumerate(model.layers):
        model.layers[i].trainable = False
    
    model.compile(optimizer='adam', loss='binary_crossentropy', metrics=['accuracy', dice_coef])
    return model

class DualUnet():
    
    def __init__(self, m1_path, m2_path, height=128, width=128, channels=3):
        
        self.inputs = tf.keras.layers.Input((height, width, channels), name='Input_dual')
        
        self.model1 = clone_model(freeze_all(tf.keras.models.load_model(m1_path)), input_tensors=self.inputs)
        self.model2 = clone_model(freeze_all(tf.keras.models.load_model(m2_path)), input_tensors=self.inputs)
        
        concat = tf.keras.layers.concatenate([self.model1.layers[-1], self.model2.layers[-1]])
        
        self.outputs = tf.keras.layers.Conv2D(1, (1, 1), kernel_initializer=tf.keras.initializers.GlorotUniform(32), activation='sigmoid')(concat)
    
    def load(self):
        model = tf.keras.Model(inputs=[self.inputs], outputs=[self.outputs])
        optimizer = 'adam'
        model.compile(optimizer=optimizer, loss='binary_crossentropy', metrics=['accuracy', dice_coef])
        self.model.summary()
        return self.model

Merging two custom Keras models into new model

0 Answers0