How to create a decoder with varying input shape but fixed output shape?

Question

I was playing around with autoencoders for mnist recently and this question appeared. "Is it possible to create a model with varying input shape but fixed output shape?"

Example:

Regularly decoder input will be e.g keras.layers.InputLayer(input_shape=(7, 7, 1)), you will have some UpSampling layers in the model to bring shape from (7,7,1) up to (28, 28, 1).

But what if decoder Input has unspecified shape?

Imagine convolutional decoder with input layer keras.layers.InputLayer(input_shape=(None, None, 1)). Input shapes for decoder maybe be different, however, decoder's output always has a fixed shape (28, 28, 1). How to build a model that will determine how to do UpSampling depending on input shape it received?

editted: Let me know if this question does not make any sense. I will delete it;)

@TomMcLean, 1 decoder will learn how to reconstruct image given different encoded features shapes — raksa, Sep 25 '22 at 13:16
"Is it possible" is not a programming question. Virtually anything is possible and the ultimate answer to this question is either "Yes" or "No". — o-90, Sep 25 '22 at 14:54

score 1 · Accepted Answer · answered Sep 25 '22 at 15:13

Conv and MaxPool layers can operate on variable size input. Dense layers need to know the size. So before them you can put a GlobalMaxPooling or similar layer. For example:

image_1 = np.random.random((1, 500, 600, 3))
image_2 = np.random.random((1, 200, 300, 3))
y = np.ones((1, 28, 28, 1))

inputs = Input(shape=(None, None, 3))
l = inputs
l = Conv2D(128, 7, activation='relu', padding='same')(l)
l = MaxPooling2D(2)(l)
l = GlobalMaxPooling2D()(l)
l = Dense(14*14, activation='relu')(l)
l = Reshape((14, 14, 1))(l)
outputs= UpSampling2D((2, 2))(l)
model = Model(inputs=inputs, outputs=outputs)
model.compile(loss='mse', optimizer='adam')
model.fit(image_1, y)
model.fit(image_2, y)
model.summary()

The model will accept variable size images, and has an output of shape (28, 28, 1). Model summary:

Model: "model"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 input_1 (InputLayer)        [(None, None, None, 3)]   0         
                                                                 
 conv2d (Conv2D)             (None, None, None, 128)   18944     
                                                                 
 max_pooling2d (MaxPooling2D  (None, None, None, 128)  0         
 )                                                               
                                                                 
 global_max_pooling2d (Globa  (None, 128)              0         
 lMaxPooling2D)                                                  
                                                                 
 dense (Dense)               (None, 196)               25284     
                                                                 
 reshape (Reshape)           (None, 14, 14, 1)         0         
                                                                 
 up_sampling2d (UpSampling2D  (None, 28, 28, 1)        0         
 )                                                               
                                                                 
=================================================================

How to create a decoder with varying input shape but fixed output shape?

1 Answers1